{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.10"
    },
    "colab": {
      "name": "DRL_single_stock_trading.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/AI4Finance-LLC/FinRL-Library/blob/master/DRL_single_stock_trading.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Idd1jem0TnST"
      },
      "source": [
        "# Deep Reinforcement Learning for Stock Trading from Scratch: Single Stock Trading\n",
        "\n",
        "Tutorials to use OpenAI DRL to trade single stock in one Jupyter Notebook | Presented at NeurIPS 2020: Deep RL Workshop\n",
        "\n",
        "* This blog is based on our paper: FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance, presented at NeurIPS 2020: Deep RL Workshop.\n",
        "* Check out medium blog for detailed explanations: \n",
        "* Please report any issues to our Github: https://github.com/AI4Finance-LLC/FinRL-Library/issues\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xD3f90UnTnSU"
      },
      "source": [
        "## Python Package Installation\n",
        "\n",
        "As a first step we check if the additional packages needed are present, if not install them. \n",
        "* Yahoo Finance API\n",
        "* pandas\n",
        "* matplotlib\n",
        "* stockstats\n",
        "* OpenAI gym\n",
        "* stable-baselines\n",
        "* tensorflow"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jeFTP-XzTnSV",
        "outputId": "3101d1db-00b8-43af-9a35-3add21c24557",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "import pkg_resources\n",
        "import pip\n",
        "installedPackages = {pkg.key for pkg in pkg_resources.working_set}\n",
        "required = {'yfinance', 'pandas', 'matplotlib', 'stockstats','stable-baselines','gym','tensorflow'}\n",
        "missing = required - installedPackages\n",
        "if missing:\n",
        "    !pip install yfinance\n",
        "    !pip install pandas\n",
        "    !pip install matplotlib\n",
        "    !pip install stockstats\n",
        "    !pip install gym\n",
        "    !pip install stable-baselines[mpi]\n",
        "    !pip install tensorflow==1.15.4\n"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting yfinance\n",
            "  Downloading https://files.pythonhosted.org/packages/7a/e8/b9d7104d3a4bf39924799067592d9e59119fcfc900a425a12e80a3123ec8/yfinance-0.1.55.tar.gz\n",
            "Requirement already satisfied: pandas>=0.24 in /usr/local/lib/python3.6/dist-packages (from yfinance) (1.1.2)\n",
            "Requirement already satisfied: numpy>=1.15 in /usr/local/lib/python3.6/dist-packages (from yfinance) (1.18.5)\n",
            "Requirement already satisfied: requests>=2.20 in /usr/local/lib/python3.6/dist-packages (from yfinance) (2.23.0)\n",
            "Requirement already satisfied: multitasking>=0.0.7 in /usr/local/lib/python3.6/dist-packages (from yfinance) (0.0.9)\n",
            "Collecting lxml>=4.5.1\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/64/28/0b761b64ecbd63d272ed0e7a6ae6e4402fc37886b59181bfdf274424d693/lxml-4.6.1-cp36-cp36m-manylinux1_x86_64.whl (5.5MB)\n",
            "\u001b[K     |████████████████████████████████| 5.5MB 3.6MB/s \n",
            "\u001b[?25hRequirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.24->yfinance) (2018.9)\n",
            "Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.24->yfinance) (2.8.1)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests>=2.20->yfinance) (3.0.4)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests>=2.20->yfinance) (2.10)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests>=2.20->yfinance) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests>=2.20->yfinance) (2020.6.20)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.6/dist-packages (from python-dateutil>=2.7.3->pandas>=0.24->yfinance) (1.15.0)\n",
            "Building wheels for collected packages: yfinance\n",
            "  Building wheel for yfinance (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for yfinance: filename=yfinance-0.1.55-py2.py3-none-any.whl size=22618 sha256=7d89f66131cbb58f2c7444aa8bc8b6b9353689d8e78f8339f3f83724acd099bd\n",
            "  Stored in directory: /root/.cache/pip/wheels/04/98/cc/2702a4242d60bdc14f48b4557c427ded1fe92aedf257d4565c\n",
            "Successfully built yfinance\n",
            "Installing collected packages: lxml, yfinance\n",
            "  Found existing installation: lxml 4.2.6\n",
            "    Uninstalling lxml-4.2.6:\n",
            "      Successfully uninstalled lxml-4.2.6\n",
            "Successfully installed lxml-4.6.1 yfinance-0.1.55\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (1.1.2)\n",
            "Requirement already satisfied: numpy>=1.15.4 in /usr/local/lib/python3.6/dist-packages (from pandas) (1.18.5)\n",
            "Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas) (2018.9)\n",
            "Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.6/dist-packages (from pandas) (2.8.1)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.6/dist-packages (from python-dateutil>=2.7.3->pandas) (1.15.0)\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (3.2.2)\n",
            "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (2.4.7)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (0.10.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (1.2.0)\n",
            "Requirement already satisfied: numpy>=1.11 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (1.18.5)\n",
            "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (2.8.1)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from cycler>=0.10->matplotlib) (1.15.0)\n",
            "Collecting stockstats\n",
            "  Downloading https://files.pythonhosted.org/packages/32/41/d3828c5bc0a262cb3112a4024108a3b019c183fa3b3078bff34bf25abf91/stockstats-0.3.2-py2.py3-none-any.whl\n",
            "Requirement already satisfied: pandas>=0.18.1 in /usr/local/lib/python3.6/dist-packages (from stockstats) (1.1.2)\n",
            "Requirement already satisfied: numpy>=1.9.2 in /usr/local/lib/python3.6/dist-packages (from stockstats) (1.18.5)\n",
            "Collecting int-date>=0.1.7\n",
            "  Downloading https://files.pythonhosted.org/packages/43/27/31803df15173ab341fe7548c14154b54227dfd8f630daa09a1c6e7db52f7/int_date-0.1.8-py2.py3-none-any.whl\n",
            "Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.18.1->stockstats) (2.8.1)\n",
            "Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.18.1->stockstats) (2018.9)\n",
            "Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.6/dist-packages (from int-date>=0.1.7->stockstats) (1.15.0)\n",
            "Installing collected packages: int-date, stockstats\n",
            "Successfully installed int-date-0.1.8 stockstats-0.3.2\n",
            "Requirement already satisfied: gym in /usr/local/lib/python3.6/dist-packages (0.17.2)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from gym) (1.4.1)\n",
            "Requirement already satisfied: cloudpickle<1.4.0,>=1.2.0 in /usr/local/lib/python3.6/dist-packages (from gym) (1.3.0)\n",
            "Requirement already satisfied: pyglet<=1.5.0,>=1.4.0 in /usr/local/lib/python3.6/dist-packages (from gym) (1.5.0)\n",
            "Requirement already satisfied: numpy>=1.10.4 in /usr/local/lib/python3.6/dist-packages (from gym) (1.18.5)\n",
            "Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from pyglet<=1.5.0,>=1.4.0->gym) (0.16.0)\n",
            "Collecting stable-baselines[mpi]\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/b0/48/d428b79bd4360727925f9fe34afeea7a9da381da3dc8748df834a349ad1d/stable_baselines-2.10.1-py3-none-any.whl (240kB)\n",
            "\u001b[K     |████████████████████████████████| 245kB 2.8MB/s \n",
            "\u001b[?25hRequirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (from stable-baselines[mpi]) (3.2.2)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.6/dist-packages (from stable-baselines[mpi]) (0.16.0)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from stable-baselines[mpi]) (1.18.5)\n",
            "Requirement already satisfied: gym[atari,classic_control]>=0.11 in /usr/local/lib/python3.6/dist-packages (from stable-baselines[mpi]) (0.17.2)\n",
            "Requirement already satisfied: opencv-python in /usr/local/lib/python3.6/dist-packages (from stable-baselines[mpi]) (4.1.2.30)\n",
            "Requirement already satisfied: cloudpickle>=0.5.5 in /usr/local/lib/python3.6/dist-packages (from stable-baselines[mpi]) (1.3.0)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (from stable-baselines[mpi]) (1.1.2)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from stable-baselines[mpi]) (1.4.1)\n",
            "Collecting mpi4py; extra == \"mpi\"\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/ec/8f/bbd8de5ba566dd77e408d8136e2bab7fdf2b97ce06cab830ba8b50a2f588/mpi4py-3.0.3.tar.gz (1.4MB)\n",
            "\u001b[K     |████████████████████████████████| 1.4MB 8.7MB/s \n",
            "\u001b[?25hRequirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->stable-baselines[mpi]) (0.10.0)\n",
            "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->stable-baselines[mpi]) (2.8.1)\n",
            "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->stable-baselines[mpi]) (2.4.7)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->stable-baselines[mpi]) (1.2.0)\n",
            "Requirement already satisfied: pyglet<=1.5.0,>=1.4.0 in /usr/local/lib/python3.6/dist-packages (from gym[atari,classic_control]>=0.11->stable-baselines[mpi]) (1.5.0)\n",
            "Requirement already satisfied: atari-py~=0.2.0; extra == \"atari\" in /usr/local/lib/python3.6/dist-packages (from gym[atari,classic_control]>=0.11->stable-baselines[mpi]) (0.2.6)\n",
            "Requirement already satisfied: Pillow; extra == \"atari\" in /usr/local/lib/python3.6/dist-packages (from gym[atari,classic_control]>=0.11->stable-baselines[mpi]) (7.0.0)\n",
            "Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas->stable-baselines[mpi]) (2018.9)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from cycler>=0.10->matplotlib->stable-baselines[mpi]) (1.15.0)\n",
            "Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from pyglet<=1.5.0,>=1.4.0->gym[atari,classic_control]>=0.11->stable-baselines[mpi]) (0.16.0)\n",
            "Building wheels for collected packages: mpi4py\n",
            "  Building wheel for mpi4py (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for mpi4py: filename=mpi4py-3.0.3-cp36-cp36m-linux_x86_64.whl size=2074501 sha256=a07c9149b0e6ef809d06789949fd347417c7b2912b70d8440ca3e9cb5e3dd987\n",
            "  Stored in directory: /root/.cache/pip/wheels/18/e0/86/2b713dd512199096012ceca61429e12b960888de59818871d6\n",
            "Successfully built mpi4py\n",
            "Installing collected packages: mpi4py, stable-baselines\n",
            "Successfully installed mpi4py-3.0.3 stable-baselines-2.10.1\n",
            "Collecting tensorflow==1.15.4\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/8e/64/7a19837dd54d3f53b1ce5ae346ab401dde9678e8f233220317000bfdb3e2/tensorflow-1.15.4-cp36-cp36m-manylinux2010_x86_64.whl (110.5MB)\n",
            "\u001b[K     |████████████████████████████████| 110.5MB 1.4MB/s \n",
            "\u001b[?25hCollecting keras-applications>=1.0.8\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/71/e3/19762fdfc62877ae9102edf6342d71b28fbfd9dea3d2f96a882ce099b03f/Keras_Applications-1.0.8-py3-none-any.whl (50kB)\n",
            "\u001b[K     |████████████████████████████████| 51kB 5.0MB/s \n",
            "\u001b[?25hCollecting gast==0.2.2\n",
            "  Downloading https://files.pythonhosted.org/packages/4e/35/11749bf99b2d4e3cceb4d55ca22590b0d7c2c62b9de38ac4a4a7f4687421/gast-0.2.2.tar.gz\n",
            "Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.4) (1.1.2)\n",
            "Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.4) (3.3.0)\n",
            "Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.4) (0.8.1)\n",
            "Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.4) (1.15.0)\n",
            "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.4) (1.1.0)\n",
            "Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.4) (0.2.0)\n",
            "Collecting tensorflow-estimator==1.15.1\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/de/62/2ee9cd74c9fa2fa450877847ba560b260f5d0fb70ee0595203082dafcc9d/tensorflow_estimator-1.15.1-py2.py3-none-any.whl (503kB)\n",
            "\u001b[K     |████████████████████████████████| 512kB 41.9MB/s \n",
            "\u001b[?25hRequirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.4) (0.10.0)\n",
            "Requirement already satisfied: wheel>=0.26; python_version >= \"3\" in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.4) (0.35.1)\n",
            "Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.4) (3.12.4)\n",
            "Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.4) (1.32.0)\n",
            "Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.4) (1.12.1)\n",
            "Requirement already satisfied: numpy<1.19.0,>=1.16.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.4) (1.18.5)\n",
            "Collecting tensorboard<1.16.0,>=1.15.0\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/1e/e9/d3d747a97f7188f48aa5eda486907f3b345cd409f0a0850468ba867db246/tensorboard-1.15.0-py3-none-any.whl (3.8MB)\n",
            "\u001b[K     |████████████████████████████████| 3.8MB 39.6MB/s \n",
            "\u001b[?25hRequirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.8->tensorflow==1.15.4) (2.10.0)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf>=3.6.1->tensorflow==1.15.4) (50.3.0)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.4) (3.2.2)\n",
            "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.4) (1.0.1)\n",
            "Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.4) (2.0.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < \"3.8\"->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.4) (3.2.0)\n",
            "Building wheels for collected packages: gast\n",
            "  Building wheel for gast (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for gast: filename=gast-0.2.2-cp36-none-any.whl size=7542 sha256=a55b678baf5d7fd287bc9ef15efbfa88a98fa4cd0b8b71a04f0bd501d4c5ea4d\n",
            "  Stored in directory: /root/.cache/pip/wheels/5c/2e/7e/a1d4d4fcebe6c381f378ce7743a3ced3699feb89bcfbdadadd\n",
            "Successfully built gast\n",
            "\u001b[31mERROR: tensorflow-probability 0.11.0 has requirement gast>=0.3.2, but you'll have gast 0.2.2 which is incompatible.\u001b[0m\n",
            "Installing collected packages: keras-applications, gast, tensorflow-estimator, tensorboard, tensorflow\n",
            "  Found existing installation: gast 0.3.3\n",
            "    Uninstalling gast-0.3.3:\n",
            "      Successfully uninstalled gast-0.3.3\n",
            "  Found existing installation: tensorflow-estimator 2.3.0\n",
            "    Uninstalling tensorflow-estimator-2.3.0:\n",
            "      Successfully uninstalled tensorflow-estimator-2.3.0\n",
            "  Found existing installation: tensorboard 2.3.0\n",
            "    Uninstalling tensorboard-2.3.0:\n",
            "      Successfully uninstalled tensorboard-2.3.0\n",
            "  Found existing installation: tensorflow 2.3.0\n",
            "    Uninstalling tensorflow-2.3.0:\n",
            "      Successfully uninstalled tensorflow-2.3.0\n",
            "Successfully installed gast-0.2.2 keras-applications-1.0.8 tensorboard-1.15.0 tensorflow-1.15.4 tensorflow-estimator-1.15.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QG17M4JwTnSZ"
      },
      "source": [
        "## Import packages"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0-0bsNMMTnSZ",
        "outputId": "8eec358b-10cd-4a36-fc8f-434ba8a586d5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        }
      },
      "source": [
        "import yfinance as yf\n",
        "from stockstats import StockDataFrame as Sdf\n",
        "\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "import gym\n",
        "from stable_baselines import PPO2, DDPG, A2C, ACKTR, TD3\n",
        "from stable_baselines import DDPG\n",
        "from stable_baselines import A2C\n",
        "from stable_baselines import SAC\n",
        "from stable_baselines.common.vec_env import DummyVecEnv\n",
        "from stable_baselines.common.policies import MlpPolicy\n"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:\n",
            "The TensorFlow contrib module will not be included in TensorFlow 2.0.\n",
            "For more information, please see:\n",
            "  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md\n",
            "  * https://github.com/tensorflow/addons\n",
            "  * https://github.com/tensorflow/io (for I/O related ops)\n",
            "If you depend on functionality not listed there, please file an issue.\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uIPbzYs1TnSd"
      },
      "source": [
        "#Diable the warnings\n",
        "import warnings\n",
        "warnings.filterwarnings('ignore')"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SBPM0sVvTnSg"
      },
      "source": [
        "# Part 1: Download Data\n",
        "Yahoo Finance is a website that provides stock data, financial news, financial reports, etc. All the data provided by Yahoo Finance is free."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JtIFikNyTnSg",
        "outputId": "f6b596ae-b91b-489c-be04-95dfcfa4a36f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Download and save the data in a pandas DataFrame:\n",
        "data_df = yf.download(\"AAPL\", start=\"2009-01-01\", end=\"2020-10-23\")"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\r[*********************100%***********************]  1 of 1 completed\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E8ZKQmw6TnSl",
        "outputId": "517ed2cf-5a8c-4319-df1c-4f11fdb2d969",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "data_df.shape"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(2973, 6)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HWxs5dmzTnSp"
      },
      "source": [
        "# reset the index, we want to use numbers instead of dates\n",
        "data_df=data_df.reset_index()"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vi6D_ED6TnSs",
        "outputId": "239bbfc5-a88d-4140-c886-6ccf5b1179c3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "data_df.head()"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Date</th>\n",
              "      <th>Open</th>\n",
              "      <th>High</th>\n",
              "      <th>Low</th>\n",
              "      <th>Close</th>\n",
              "      <th>Adj Close</th>\n",
              "      <th>Volume</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2009-01-02</td>\n",
              "      <td>3.067143</td>\n",
              "      <td>3.251429</td>\n",
              "      <td>3.041429</td>\n",
              "      <td>3.241071</td>\n",
              "      <td>2.800736</td>\n",
              "      <td>746015200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2009-01-05</td>\n",
              "      <td>3.327500</td>\n",
              "      <td>3.435000</td>\n",
              "      <td>3.311071</td>\n",
              "      <td>3.377857</td>\n",
              "      <td>2.918938</td>\n",
              "      <td>1181608400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2009-01-06</td>\n",
              "      <td>3.426786</td>\n",
              "      <td>3.470357</td>\n",
              "      <td>3.299643</td>\n",
              "      <td>3.322143</td>\n",
              "      <td>2.870794</td>\n",
              "      <td>1289310400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2009-01-07</td>\n",
              "      <td>3.278929</td>\n",
              "      <td>3.303571</td>\n",
              "      <td>3.223572</td>\n",
              "      <td>3.250357</td>\n",
              "      <td>2.808761</td>\n",
              "      <td>753048800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2009-01-08</td>\n",
              "      <td>3.229643</td>\n",
              "      <td>3.326786</td>\n",
              "      <td>3.215714</td>\n",
              "      <td>3.310714</td>\n",
              "      <td>2.860918</td>\n",
              "      <td>673500800</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "        Date      Open      High       Low     Close  Adj Close      Volume\n",
              "0 2009-01-02  3.067143  3.251429  3.041429  3.241071   2.800736   746015200\n",
              "1 2009-01-05  3.327500  3.435000  3.311071  3.377857   2.918938  1181608400\n",
              "2 2009-01-06  3.426786  3.470357  3.299643  3.322143   2.870794  1289310400\n",
              "3 2009-01-07  3.278929  3.303571  3.223572  3.250357   2.808761   753048800\n",
              "4 2009-01-08  3.229643  3.326786  3.215714  3.310714   2.860918   673500800"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KX3UrIu6TnSv",
        "outputId": "151ed16f-a2e5-4d2c-bbfa-146d4a6306bb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "data_df.columns"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Index(['Date', 'Open', 'High', 'Low', 'Close', 'Adj Close', 'Volume'], dtype='object')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UyN2BSsGTnSy"
      },
      "source": [
        "# convert the column names to standardized names\n",
        "data_df.columns = ['datadate','open','high','low','close','adjcp','volume']"
      ],
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "r_g0AmjHTnS1"
      },
      "source": [
        "# save the data to a csv file in your current folder\n",
        "#data_df.to_csv('AAPL_2009_2020.csv')"
      ],
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oWiqgpLzTnS3"
      },
      "source": [
        "# Part 2: Preprocess Data\n",
        "Data preprocessing is a crucial step for training a high quality machine learning model. We need to check for missing data and do feature engineering in order to convert the data into a model-ready state."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7EQlXSl1TnS4",
        "outputId": "b090b592-75a8-4962-f674-e047079b2147",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# check missing data \n",
        "data_df.isnull().values.any()"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TzrRbc1jTnS6"
      },
      "source": [
        "# calculate technical indicators like MACD\n",
        "stock = Sdf.retype(data_df.copy())\n",
        "# we need to use adjusted close price instead of close price\n",
        "stock['close'] = stock['adjcp']\n",
        "data_df['macd'] = stock['macd']"
      ],
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Wersy6QKTnS9",
        "outputId": "bc418ce7-40bb-4a82-d80f-6433f250cd0e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# check missing data again\n",
        "data_df.isnull().values.any()"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7ZqXSKC6TnTB",
        "outputId": "3ab8c015-a390-4593-a9a5-cefc38e2a367",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "data_df.head()"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>datadate</th>\n",
              "      <th>open</th>\n",
              "      <th>high</th>\n",
              "      <th>low</th>\n",
              "      <th>close</th>\n",
              "      <th>adjcp</th>\n",
              "      <th>volume</th>\n",
              "      <th>macd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2009-01-02</td>\n",
              "      <td>3.067143</td>\n",
              "      <td>3.251429</td>\n",
              "      <td>3.041429</td>\n",
              "      <td>3.241071</td>\n",
              "      <td>2.800736</td>\n",
              "      <td>746015200</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2009-01-05</td>\n",
              "      <td>3.327500</td>\n",
              "      <td>3.435000</td>\n",
              "      <td>3.311071</td>\n",
              "      <td>3.377857</td>\n",
              "      <td>2.918938</td>\n",
              "      <td>1181608400</td>\n",
              "      <td>0.002652</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2009-01-06</td>\n",
              "      <td>3.426786</td>\n",
              "      <td>3.470357</td>\n",
              "      <td>3.299643</td>\n",
              "      <td>3.322143</td>\n",
              "      <td>2.870794</td>\n",
              "      <td>1289310400</td>\n",
              "      <td>0.001886</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2009-01-07</td>\n",
              "      <td>3.278929</td>\n",
              "      <td>3.303571</td>\n",
              "      <td>3.223572</td>\n",
              "      <td>3.250357</td>\n",
              "      <td>2.808761</td>\n",
              "      <td>753048800</td>\n",
              "      <td>-0.000748</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2009-01-08</td>\n",
              "      <td>3.229643</td>\n",
              "      <td>3.326786</td>\n",
              "      <td>3.215714</td>\n",
              "      <td>3.310714</td>\n",
              "      <td>2.860918</td>\n",
              "      <td>673500800</td>\n",
              "      <td>-0.000088</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    datadate      open      high  ...     adjcp      volume      macd\n",
              "0 2009-01-02  3.067143  3.251429  ...  2.800736   746015200  0.000000\n",
              "1 2009-01-05  3.327500  3.435000  ...  2.918938  1181608400  0.002652\n",
              "2 2009-01-06  3.426786  3.470357  ...  2.870794  1289310400  0.001886\n",
              "3 2009-01-07  3.278929  3.303571  ...  2.808761   753048800 -0.000748\n",
              "4 2009-01-08  3.229643  3.326786  ...  2.860918   673500800 -0.000088\n",
              "\n",
              "[5 rows x 8 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kIIo8WNgTnTF"
      },
      "source": [
        "#data_df=data_df.fillna(method='bfill')"
      ],
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cStYZrakTnTI"
      },
      "source": [
        "# Note that I always use a copy of the original data to try it track step by step.\n",
        "data_clean = data_df.copy()"
      ],
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MSAe2FGdTnTK",
        "outputId": "e4369ed1-c9e8-4555-f95c-1811f018be8b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "data_clean.head()"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>datadate</th>\n",
              "      <th>open</th>\n",
              "      <th>high</th>\n",
              "      <th>low</th>\n",
              "      <th>close</th>\n",
              "      <th>adjcp</th>\n",
              "      <th>volume</th>\n",
              "      <th>macd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2009-01-02</td>\n",
              "      <td>3.067143</td>\n",
              "      <td>3.251429</td>\n",
              "      <td>3.041429</td>\n",
              "      <td>3.241071</td>\n",
              "      <td>2.800736</td>\n",
              "      <td>746015200</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2009-01-05</td>\n",
              "      <td>3.327500</td>\n",
              "      <td>3.435000</td>\n",
              "      <td>3.311071</td>\n",
              "      <td>3.377857</td>\n",
              "      <td>2.918938</td>\n",
              "      <td>1181608400</td>\n",
              "      <td>0.002652</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2009-01-06</td>\n",
              "      <td>3.426786</td>\n",
              "      <td>3.470357</td>\n",
              "      <td>3.299643</td>\n",
              "      <td>3.322143</td>\n",
              "      <td>2.870794</td>\n",
              "      <td>1289310400</td>\n",
              "      <td>0.001886</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2009-01-07</td>\n",
              "      <td>3.278929</td>\n",
              "      <td>3.303571</td>\n",
              "      <td>3.223572</td>\n",
              "      <td>3.250357</td>\n",
              "      <td>2.808761</td>\n",
              "      <td>753048800</td>\n",
              "      <td>-0.000748</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2009-01-08</td>\n",
              "      <td>3.229643</td>\n",
              "      <td>3.326786</td>\n",
              "      <td>3.215714</td>\n",
              "      <td>3.310714</td>\n",
              "      <td>2.860918</td>\n",
              "      <td>673500800</td>\n",
              "      <td>-0.000088</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    datadate      open      high  ...     adjcp      volume      macd\n",
              "0 2009-01-02  3.067143  3.251429  ...  2.800736   746015200  0.000000\n",
              "1 2009-01-05  3.327500  3.435000  ...  2.918938  1181608400  0.002652\n",
              "2 2009-01-06  3.426786  3.470357  ...  2.870794  1289310400  0.001886\n",
              "3 2009-01-07  3.278929  3.303571  ...  2.808761   753048800 -0.000748\n",
              "4 2009-01-08  3.229643  3.326786  ...  2.860918   673500800 -0.000088\n",
              "\n",
              "[5 rows x 8 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5L_4KOv_TnTN",
        "outputId": "b109bd3e-7037-4f9d-86f0-48ceadc9a6e0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "data_clean.tail()"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>datadate</th>\n",
              "      <th>open</th>\n",
              "      <th>high</th>\n",
              "      <th>low</th>\n",
              "      <th>close</th>\n",
              "      <th>adjcp</th>\n",
              "      <th>volume</th>\n",
              "      <th>macd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2968</th>\n",
              "      <td>2020-10-16</td>\n",
              "      <td>121.279999</td>\n",
              "      <td>121.550003</td>\n",
              "      <td>118.809998</td>\n",
              "      <td>119.019997</td>\n",
              "      <td>119.019997</td>\n",
              "      <td>115393800</td>\n",
              "      <td>1.645568</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2969</th>\n",
              "      <td>2020-10-19</td>\n",
              "      <td>119.959999</td>\n",
              "      <td>120.419998</td>\n",
              "      <td>115.660004</td>\n",
              "      <td>115.980003</td>\n",
              "      <td>115.980003</td>\n",
              "      <td>120639300</td>\n",
              "      <td>1.346273</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2970</th>\n",
              "      <td>2020-10-20</td>\n",
              "      <td>116.199997</td>\n",
              "      <td>118.980003</td>\n",
              "      <td>115.629997</td>\n",
              "      <td>117.510002</td>\n",
              "      <td>117.510002</td>\n",
              "      <td>124423700</td>\n",
              "      <td>1.218491</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2971</th>\n",
              "      <td>2020-10-21</td>\n",
              "      <td>116.669998</td>\n",
              "      <td>118.709999</td>\n",
              "      <td>116.449997</td>\n",
              "      <td>116.870003</td>\n",
              "      <td>116.870003</td>\n",
              "      <td>89946000</td>\n",
              "      <td>1.053437</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2972</th>\n",
              "      <td>2020-10-22</td>\n",
              "      <td>117.449997</td>\n",
              "      <td>118.040001</td>\n",
              "      <td>114.589996</td>\n",
              "      <td>115.750000</td>\n",
              "      <td>115.750000</td>\n",
              "      <td>101988000</td>\n",
              "      <td>0.822772</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       datadate        open        high  ...       adjcp     volume      macd\n",
              "2968 2020-10-16  121.279999  121.550003  ...  119.019997  115393800  1.645568\n",
              "2969 2020-10-19  119.959999  120.419998  ...  115.980003  120639300  1.346273\n",
              "2970 2020-10-20  116.199997  118.980003  ...  117.510002  124423700  1.218491\n",
              "2971 2020-10-21  116.669998  118.709999  ...  116.870003   89946000  1.053437\n",
              "2972 2020-10-22  117.449997  118.040001  ...  115.750000  101988000  0.822772\n",
              "\n",
              "[5 rows x 8 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bwLhXo1cTnTQ"
      },
      "source": [
        "# Part 3: Design Environment\n",
        "Considering the stochastic and interactive nature of the automated stock trading tasks, a financial task is modeled as a Markov Decision Process (MDP) problem. The training process involves observing stock price change, taking an action and reward's calculation to have the agent adjusting its strategy accordingly. By interacting with the environment, the trading agent will derive a trading strategy with the maximized rewards as time proceeds.\n",
        "\n",
        "Our trading environments, based on OpenAI Gym framework, simulate live stock markets with real market data according to the principle of time-driven simulation.\n",
        "\n",
        "The action space describes the allowed actions that the agent interacts with the environment. Normally, action a includes three actions: {-1, 0, 1}, where -1, 0, 1 represent selling, holding, and buying one share. Also, an action can be carried upon multiple shares. We use an action space {-k,…,-1, 0, 1, …, k}, where k denotes the number of shares to buy and -k denotes the number of shares to sell. For example, \"Buy 10 shares of AAPL\" or \"Sell 10 shares of AAPL\" are 10 or -10, respectively. The continuous action space needs to be normalized to [-1, 1], since the policy is defined on a Gaussian distribution, which needs to be normalized and symmetric."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5qqW-mnSTnTQ"
      },
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "from gym.utils import seeding\n",
        "import gym\n",
        "from gym import spaces\n",
        "import matplotlib\n",
        "matplotlib.use('Agg')\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "# Global variables\n",
        "HMAX_NORMALIZE = 200\n",
        "INITIAL_ACCOUNT_BALANCE=100000\n",
        "STOCK_DIM = 1\n",
        "\n",
        "# transaction fee: 1/1000 reasonable percentage\n",
        "TRANSACTION_FEE_PERCENT = 0.001\n",
        "# REWARD_SCALING = 1e-3\n",
        "\n",
        "\n",
        "class SingleStockEnv(gym.Env):\n",
        "    \"\"\"A stock trading environment for OpenAI gym\"\"\"\n",
        "    metadata = {'render.modes': ['human']}\n",
        "\n",
        "    def __init__(self, df,day = 0):\n",
        "        #super(StockEnv, self).__init__()\n",
        "        # date increment\n",
        "        self.day = day\n",
        "        self.df = df\n",
        "        # action_space normalization and the shape is STOCK_DIM\n",
        "        self.action_space = spaces.Box(low = -1, high = 1,shape = (STOCK_DIM,)) \n",
        "        # Shape = 4: [Current Balance]+[prices]+[owned shares] +[macd] \n",
        "        self.observation_space = spaces.Box(low=0, high=np.inf, shape = (4,))\n",
        "        # load data from a pandas dataframe\n",
        "        self.data = self.df.loc[self.day,:]\n",
        "        # termination\n",
        "        self.terminal = False  \n",
        "        # save the total number of trades\n",
        "        self.trades = 0\n",
        "        # initalize state\n",
        "        self.state = [INITIAL_ACCOUNT_BALANCE] + \\\n",
        "                      [self.data.adjcp] + \\\n",
        "                      [0]*STOCK_DIM + \\\n",
        "                      [self.data.macd] \n",
        "        # initialize reward and cost\n",
        "        self.reward = 0\n",
        "        self.cost = 0\n",
        "        \n",
        "        # memorize the total value, total rewards\n",
        "        self.asset_memory = [INITIAL_ACCOUNT_BALANCE]\n",
        "        self.rewards_memory = []\n",
        "\n",
        "    def _sell_stock(self, index, action):\n",
        "        # perform sell action based on the sign of the action\n",
        "        if self.state[index+STOCK_DIM+1] > 0:\n",
        "            # update balance\n",
        "            self.state[0] += \\\n",
        "            self.state[index+1]*min(abs(action),self.state[index+STOCK_DIM+1]) * \\\n",
        "             (1- TRANSACTION_FEE_PERCENT)\n",
        "            # update held shares\n",
        "            self.state[index+STOCK_DIM+1] -= min(abs(action), self.state[index+STOCK_DIM+1])\n",
        "            # update transaction costs\n",
        "            self.cost +=self.state[index+1]*min(abs(action),self.state[index+STOCK_DIM+1]) * \\\n",
        "             TRANSACTION_FEE_PERCENT\n",
        "            self.trades+=1\n",
        "        else:\n",
        "            pass\n",
        "            \n",
        "    def _buy_stock(self, index, action):\n",
        "        # perform buy action based on the sign of the action\n",
        "        available_amount = self.state[0] // self.state[index+1]\n",
        "        #update balance\n",
        "        self.state[0] -= self.state[index+1]*min(available_amount, action)* \\\n",
        "                          (1+ TRANSACTION_FEE_PERCENT)\n",
        "        # update held shares\n",
        "        self.state[index+STOCK_DIM+1] += min(available_amount, action)\n",
        "        # update transaction costs\n",
        "        self.cost+=self.state[index+1]*min(available_amount, action)* \\\n",
        "                          TRANSACTION_FEE_PERCENT\n",
        "        self.trades+=1\n",
        "        \n",
        "    def step(self, actions):\n",
        "        self.terminal = self.day >= len(self.df.index.unique())-1\n",
        "\n",
        "        if self.terminal:\n",
        "            plt.plot(self.asset_memory,'r')\n",
        "            plt.savefig('account_value.png')\n",
        "            plt.close()\n",
        "            \n",
        "            end_total_asset = self.state[0]+ \\\n",
        "            sum(np.array(self.state[1:(STOCK_DIM+1)])*np.array(self.state[(STOCK_DIM+1):(STOCK_DIM*2+1)]))\n",
        "            print(\"previous_total_asset:{}\".format(self.asset_memory[0])) \n",
        "            print(\"end_total_asset:{}\".format(end_total_asset))\n",
        "            \n",
        "            df_total_value = pd.DataFrame(self.asset_memory)\n",
        "            df_total_value.to_csv('account_value.csv')\n",
        "            print(\"total_reward:{}\".format(self.state[0]+sum(np.array(self.state[1:(STOCK_DIM+1)])*np.array(self.state[(STOCK_DIM+1):(STOCK_DIM*2+1)]))- INITIAL_ACCOUNT_BALANCE ))\n",
        "            print(\"total_cost: \", self.cost)\n",
        "            print(\"total trades: \", self.trades)\n",
        "            \n",
        "            df_total_value.columns = ['account_value']\n",
        "            df_total_value['daily_return']=df_total_value.pct_change(1)\n",
        "            \n",
        "            if df_total_value['daily_return'].std()!=0:\n",
        "                sharpe = (252**0.5)*df_total_value['daily_return'].mean()/ \\\n",
        "                      df_total_value['daily_return'].std()\n",
        "                print(\"Sharpe: \",sharpe)\n",
        "            df_rewards = pd.DataFrame(self.rewards_memory)\n",
        "            df_rewards.to_csv('account_rewards.csv')\n",
        "            return self.state, self.reward, self.terminal,{}\n",
        "\n",
        "        else:\n",
        "            \n",
        "            # actions are the shares we need to buy, hold, or sell\n",
        "            actions = actions * HMAX_NORMALIZE\n",
        "            # calculate begining total asset\n",
        "            begin_total_asset = self.state[0]+ \\\n",
        "            sum(np.array(self.state[1:(STOCK_DIM+1)])*np.array(self.state[(STOCK_DIM+1):(STOCK_DIM*2+1)]))\n",
        "            \n",
        "            # perform buy or sell action\n",
        "            argsort_actions = np.argsort(actions)\n",
        "            sell_index = argsort_actions[:np.where(actions < 0)[0].shape[0]]\n",
        "            buy_index = argsort_actions[::-1][:np.where(actions > 0)[0].shape[0]]\n",
        "\n",
        "            for index in sell_index:\n",
        "                # print('take sell action'.format(actions[index]))\n",
        "                self._sell_stock(index, actions[index])\n",
        "\n",
        "            for index in buy_index:\n",
        "                # print('take buy action: {}'.format(actions[index]))\n",
        "                self._buy_stock(index, actions[index])\n",
        "            \n",
        "            # update data, walk a step s'\n",
        "            self.day += 1\n",
        "            self.data = self.df.loc[self.day,:]         \n",
        "            #load next state\n",
        "            self.state =  [self.state[0]] + \\\n",
        "                          [self.data.adjcp] + \\\n",
        "                          list(self.state[(STOCK_DIM+1):(STOCK_DIM*2+1)]) +\\\n",
        "                          [self.data.macd]\n",
        "                        \n",
        "            # calculate the end total asset\n",
        "            end_total_asset = self.state[0]+ \\\n",
        "            sum(np.array(self.state[1:(STOCK_DIM+1)])*np.array(self.state[(STOCK_DIM+1):(STOCK_DIM*2+1)]))\n",
        "            self.reward = end_total_asset - begin_total_asset  \n",
        "            self.rewards_memory.append(self.reward)\n",
        "            #self.reward = self.reward * REWARD_SCALING\n",
        "            self.asset_memory.append(end_total_asset)\n",
        "\n",
        "        return self.state, self.reward, self.terminal, {}\n",
        "\n",
        "    def reset(self):\n",
        "        self.asset_memory = [INITIAL_ACCOUNT_BALANCE]\n",
        "        self.day = 0\n",
        "        self.data = self.df.loc[self.day,:]\n",
        "        self.cost = 0\n",
        "        self.trades = 0\n",
        "        self.terminal = False \n",
        "        self.rewards_memory = []\n",
        "        #initiate state\n",
        "        self.state = [INITIAL_ACCOUNT_BALANCE] + \\\n",
        "                      [self.data.adjcp] + \\\n",
        "                      [0]*STOCK_DIM + \\\n",
        "                      [self.data.macd]\n",
        "        return self.state\n",
        "    \n",
        "    def render(self, mode='human'):\n",
        "        return self.state\n",
        "\n",
        "    def _seed(self, seed=None):\n",
        "        self.np_random, seed = seeding.np_random(seed)\n",
        "        return [seed]"
      ],
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jIGqfq0FTnTT"
      },
      "source": [
        ""
      ],
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mdnzYtM1TnTW"
      },
      "source": [
        "# Part 4: Implement DRL Algorithms\n",
        "The implementation of the DRL algorithms are based on OpenAI Baselines and Stable Baselines. Stable Baselines is a fork of OpenAI Baselines, with a major structural refactoring, and code cleanups."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fxvtFex3TnTW"
      },
      "source": [
        "### Training data: 2009-01-01 to 2018-12-31"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BRFIZDw8TnTX"
      },
      "source": [
        "train = data_clean[(data_clean.datadate>='2009-01-01') & (data_clean.datadate<'2019-01-01')]\n",
        "# the index needs to start from 0\n",
        "train=train.reset_index(drop=True)"
      ],
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hdjTsY4vTnTZ",
        "outputId": "785ce8e6-afe4-487c-dee5-f5c56952a17b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "train.head()"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>datadate</th>\n",
              "      <th>open</th>\n",
              "      <th>high</th>\n",
              "      <th>low</th>\n",
              "      <th>close</th>\n",
              "      <th>adjcp</th>\n",
              "      <th>volume</th>\n",
              "      <th>macd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2009-01-02</td>\n",
              "      <td>3.067143</td>\n",
              "      <td>3.251429</td>\n",
              "      <td>3.041429</td>\n",
              "      <td>3.241071</td>\n",
              "      <td>2.800736</td>\n",
              "      <td>746015200</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2009-01-05</td>\n",
              "      <td>3.327500</td>\n",
              "      <td>3.435000</td>\n",
              "      <td>3.311071</td>\n",
              "      <td>3.377857</td>\n",
              "      <td>2.918938</td>\n",
              "      <td>1181608400</td>\n",
              "      <td>0.002652</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2009-01-06</td>\n",
              "      <td>3.426786</td>\n",
              "      <td>3.470357</td>\n",
              "      <td>3.299643</td>\n",
              "      <td>3.322143</td>\n",
              "      <td>2.870794</td>\n",
              "      <td>1289310400</td>\n",
              "      <td>0.001886</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2009-01-07</td>\n",
              "      <td>3.278929</td>\n",
              "      <td>3.303571</td>\n",
              "      <td>3.223572</td>\n",
              "      <td>3.250357</td>\n",
              "      <td>2.808761</td>\n",
              "      <td>753048800</td>\n",
              "      <td>-0.000748</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2009-01-08</td>\n",
              "      <td>3.229643</td>\n",
              "      <td>3.326786</td>\n",
              "      <td>3.215714</td>\n",
              "      <td>3.310714</td>\n",
              "      <td>2.860918</td>\n",
              "      <td>673500800</td>\n",
              "      <td>-0.000088</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    datadate      open      high  ...     adjcp      volume      macd\n",
              "0 2009-01-02  3.067143  3.251429  ...  2.800736   746015200  0.000000\n",
              "1 2009-01-05  3.327500  3.435000  ...  2.918938  1181608400  0.002652\n",
              "2 2009-01-06  3.426786  3.470357  ...  2.870794  1289310400  0.001886\n",
              "3 2009-01-07  3.278929  3.303571  ...  2.808761   753048800 -0.000748\n",
              "4 2009-01-08  3.229643  3.326786  ...  2.860918   673500800 -0.000088\n",
              "\n",
              "[5 rows x 8 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zD1NHzGyTnTc"
      },
      "source": [
        "### Model Training: 4 models, PPO A2C, DDPG, TD3"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y9CM5DeIr9GC"
      },
      "source": [
        "### Model 1: PPO"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "UJs0N4dGTnTd",
        "outputId": "2b896ae5-dab6-4e53-8c81-a7739b5d2924",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "#tensorboard --logdir ./single_stock_tensorboard/\n",
        "env_train = DummyVecEnv([lambda: SingleStockEnv(train)])\n",
        "model_ppo = PPO2('MlpPolicy', env_train, tensorboard_log=\"./single_stock_trading_2_tensorboard/\")\n",
        "model_ppo.learn(total_timesteps=100000,tb_log_name=\"run_aapl_ppo\")\n",
        "#model.save('AAPL_ppo_100k')\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/common/tf_util.py:191: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/common/tf_util.py:200: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/common/policies.py:116: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/common/input.py:25: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/common/policies.py:561: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use keras.layers.flatten instead.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/layers/core.py:332: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Please use `layer.__call__` method instead.\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/common/tf_layers.py:123: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/common/distributions.py:418: The name tf.random_normal is deprecated. Please use tf.random.normal instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/ppo2/ppo2.py:190: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/ppo2/ppo2.py:198: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/math_grad.py:1424: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/ppo2/ppo2.py:206: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/ppo2/ppo2.py:240: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/ppo2/ppo2.py:242: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/common/base_class.py:1169: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
            "\n",
            "previous_total_asset:100000\n",
            "end_total_asset:156098.18901026587\n",
            "total_reward:56098.189010265865\n",
            "total_cost:  6341.287377798898\n",
            "total trades:  2491\n",
            "Sharpe:  0.4187558739862496\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/common/tf_util.py:502: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
            "\n",
            "previous_total_asset:100000\n",
            "end_total_asset:94446.32340155944\n",
            "total_reward:-5553.676598440565\n",
            "total_cost:  2929.111106588759\n",
            "total trades:  1887\n",
            "Sharpe:  -0.4449912464199369\n",
            "previous_total_asset:100000\n",
            "end_total_asset:106926.53386862631\n",
            "total_reward:6926.533868626313\n",
            "total_cost:  5599.619928583559\n",
            "total trades:  2210\n",
            "Sharpe:  0.13181291830866868\n",
            "previous_total_asset:100000\n",
            "end_total_asset:210752.72119495703\n",
            "total_reward:110752.72119495703\n",
            "total_cost:  6167.86721222417\n",
            "total trades:  2436\n",
            "Sharpe:  0.5912451258666666\n",
            "previous_total_asset:100000\n",
            "end_total_asset:266216.10048735235\n",
            "total_reward:166216.10048735235\n",
            "total_cost:  6268.2672412182\n",
            "total trades:  2506\n",
            "Sharpe:  0.6394795415623377\n",
            "previous_total_asset:100000\n",
            "end_total_asset:553507.7849352992\n",
            "total_reward:453507.7849352992\n",
            "total_cost:  6205.074570567108\n",
            "total trades:  2514\n",
            "Sharpe:  0.869382821594984\n",
            "previous_total_asset:100000\n",
            "end_total_asset:525815.480627327\n",
            "total_reward:425815.480627327\n",
            "total_cost:  6268.937668332546\n",
            "total trades:  2515\n",
            "Sharpe:  0.8563922247597354\n",
            "previous_total_asset:100000\n",
            "end_total_asset:579948.3273711266\n",
            "total_reward:479948.3273711266\n",
            "total_cost:  6264.915418211092\n",
            "total trades:  2515\n",
            "Sharpe:  0.8812822887651185\n",
            "previous_total_asset:100000\n",
            "end_total_asset:601018.8610150499\n",
            "total_reward:501018.86101504986\n",
            "total_cost:  6071.15111291963\n",
            "total trades:  2514\n",
            "Sharpe:  0.9042371048628427\n",
            "previous_total_asset:100000\n",
            "end_total_asset:501034.1646295194\n",
            "total_reward:401034.1646295194\n",
            "total_cost:  6162.337433171433\n",
            "total trades:  2508\n",
            "Sharpe:  0.830281620872439\n",
            "previous_total_asset:100000\n",
            "end_total_asset:457905.0789816795\n",
            "total_reward:357905.0789816795\n",
            "total_cost:  6250.532500542996\n",
            "total trades:  2514\n",
            "Sharpe:  0.8257661702659089\n",
            "previous_total_asset:100000\n",
            "end_total_asset:265638.6391078079\n",
            "total_reward:165638.6391078079\n",
            "total_cost:  6734.51020678946\n",
            "total trades:  2509\n",
            "Sharpe:  0.7966543085095101\n",
            "previous_total_asset:100000\n",
            "end_total_asset:176177.60613806313\n",
            "total_reward:76177.60613806313\n",
            "total_cost:  5210.800510190631\n",
            "total trades:  2372\n",
            "Sharpe:  0.6105657641565995\n",
            "previous_total_asset:100000\n",
            "end_total_asset:119068.00619850191\n",
            "total_reward:19068.006198501913\n",
            "total_cost:  5449.554538212252\n",
            "total trades:  2340\n",
            "Sharpe:  0.4782932891593345\n",
            "previous_total_asset:100000\n",
            "end_total_asset:212956.67157340347\n",
            "total_reward:112956.67157340347\n",
            "total_cost:  6368.901746125352\n",
            "total trades:  2475\n",
            "Sharpe:  0.6871472400668093\n",
            "previous_total_asset:100000\n",
            "end_total_asset:395805.73324403726\n",
            "total_reward:295805.73324403726\n",
            "total_cost:  6537.4196105855235\n",
            "total trades:  2511\n",
            "Sharpe:  0.8052338342289653\n",
            "previous_total_asset:100000\n",
            "end_total_asset:604812.5459749429\n",
            "total_reward:504812.54597494286\n",
            "total_cost:  6218.48193137824\n",
            "total trades:  2513\n",
            "Sharpe:  0.9101661298526685\n",
            "previous_total_asset:100000\n",
            "end_total_asset:582116.5546294623\n",
            "total_reward:482116.5546294623\n",
            "total_cost:  6211.65512132105\n",
            "total trades:  2515\n",
            "Sharpe:  0.9192984163452886\n",
            "previous_total_asset:100000\n",
            "end_total_asset:526174.6858831241\n",
            "total_reward:426174.6858831241\n",
            "total_cost:  6416.375233592592\n",
            "total trades:  2515\n",
            "Sharpe:  0.8856642983091633\n",
            "previous_total_asset:100000\n",
            "end_total_asset:553836.0514161409\n",
            "total_reward:453836.05141614086\n",
            "total_cost:  6567.770799179734\n",
            "total trades:  2515\n",
            "Sharpe:  0.8855059161955452\n",
            "previous_total_asset:100000\n",
            "end_total_asset:616036.9849392112\n",
            "total_reward:516036.98493921116\n",
            "total_cost:  6652.36958320929\n",
            "total trades:  2512\n",
            "Sharpe:  0.9329275065056303\n",
            "previous_total_asset:100000\n",
            "end_total_asset:568439.2268575617\n",
            "total_reward:468439.22685756173\n",
            "total_cost:  6487.975412838857\n",
            "total trades:  2515\n",
            "Sharpe:  0.9065578971932241\n",
            "previous_total_asset:100000\n",
            "end_total_asset:541432.7291863837\n",
            "total_reward:441432.72918638366\n",
            "total_cost:  6154.1131312864945\n",
            "total trades:  2515\n",
            "Sharpe:  0.875571568948455\n",
            "previous_total_asset:100000\n",
            "end_total_asset:769749.4838137159\n",
            "total_reward:669749.4838137159\n",
            "total_cost:  5731.190934403671\n",
            "total trades:  2515\n",
            "Sharpe:  0.9634334035816738\n",
            "previous_total_asset:100000\n",
            "end_total_asset:746253.3798512011\n",
            "total_reward:646253.3798512011\n",
            "total_cost:  5540.084347681433\n",
            "total trades:  2514\n",
            "Sharpe:  0.9554523241222211\n",
            "previous_total_asset:100000\n",
            "end_total_asset:702178.8201897977\n",
            "total_reward:602178.8201897977\n",
            "total_cost:  5961.959511756423\n",
            "total trades:  2515\n",
            "Sharpe:  0.9397244292873406\n",
            "previous_total_asset:100000\n",
            "end_total_asset:741096.6667565618\n",
            "total_reward:641096.6667565618\n",
            "total_cost:  5986.884645727058\n",
            "total trades:  2514\n",
            "Sharpe:  0.9557487033016611\n",
            "previous_total_asset:100000\n",
            "end_total_asset:729812.9256538266\n",
            "total_reward:629812.9256538266\n",
            "total_cost:  6011.705168113822\n",
            "total trades:  2513\n",
            "Sharpe:  0.9538379004377312\n",
            "previous_total_asset:100000\n",
            "end_total_asset:758236.5753620398\n",
            "total_reward:658236.5753620398\n",
            "total_cost:  6181.802640711599\n",
            "total trades:  2513\n",
            "Sharpe:  0.9636495498888424\n",
            "previous_total_asset:100000\n",
            "end_total_asset:772840.3734990709\n",
            "total_reward:672840.3734990709\n",
            "total_cost:  5596.33548413703\n",
            "total trades:  2512\n",
            "Sharpe:  0.9681238454395429\n",
            "previous_total_asset:100000\n",
            "end_total_asset:741677.1106373993\n",
            "total_reward:641677.1106373993\n",
            "total_cost:  5955.182698275548\n",
            "total trades:  2514\n",
            "Sharpe:  0.9595399727490882\n",
            "previous_total_asset:100000\n",
            "end_total_asset:764805.0461015248\n",
            "total_reward:664805.0461015248\n",
            "total_cost:  5891.292651756751\n",
            "total trades:  2515\n",
            "Sharpe:  0.9661904649783544\n",
            "previous_total_asset:100000\n",
            "end_total_asset:789673.3483401105\n",
            "total_reward:689673.3483401105\n",
            "total_cost:  5926.372793171938\n",
            "total trades:  2515\n",
            "Sharpe:  0.9741742994885249\n",
            "previous_total_asset:100000\n",
            "end_total_asset:741254.787990579\n",
            "total_reward:641254.787990579\n",
            "total_cost:  5821.799027481282\n",
            "total trades:  2515\n",
            "Sharpe:  0.9576394615424657\n",
            "previous_total_asset:100000\n",
            "end_total_asset:731611.5428951988\n",
            "total_reward:631611.5428951988\n",
            "total_cost:  6097.654599630185\n",
            "total trades:  2515\n",
            "Sharpe:  0.9514027303396922\n",
            "previous_total_asset:100000\n",
            "end_total_asset:724101.1807242595\n",
            "total_reward:624101.1807242595\n",
            "total_cost:  6112.808160692618\n",
            "total trades:  2513\n",
            "Sharpe:  0.9468299402074484\n",
            "previous_total_asset:100000\n",
            "end_total_asset:759113.1677076282\n",
            "total_reward:659113.1677076282\n",
            "total_cost:  6216.473084561016\n",
            "total trades:  2515\n",
            "Sharpe:  0.9682276096737484\n",
            "previous_total_asset:100000\n",
            "end_total_asset:740835.2222805698\n",
            "total_reward:640835.2222805698\n",
            "total_cost:  5994.8131451707895\n",
            "total trades:  2515\n",
            "Sharpe:  0.9614725290080312\n",
            "previous_total_asset:100000\n",
            "end_total_asset:757197.6833960349\n",
            "total_reward:657197.6833960349\n",
            "total_cost:  5919.665946050203\n",
            "total trades:  2515\n",
            "Sharpe:  0.964978933627224\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<stable_baselines.ppo2.ppo2.PPO2 at 0x7f19fda01898>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9upN8FI2r_X1"
      },
      "source": [
        "### Model 2: DDPG"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "FdtXfSxpTnTf",
        "outputId": "0bcfb0cd-4872-4f86-c8f5-016c535cf0e1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "#tensorboard --logdir ./single_stock_tensorboard/\n",
        "env_train = DummyVecEnv([lambda: SingleStockEnv(train)])\n",
        "model_ddpg = DDPG('MlpPolicy', env_train, tensorboard_log=\"./single_stock_trading_2_tensorboard/\")\n",
        "model_ddpg.learn(total_timesteps=100000, tb_log_name=\"run_aapl_ddpg\")\n",
        "#model.save('AAPL_ddpg_50k')\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "previous_total_asset:100000\n",
            "end_total_asset:381113.46135500714\n",
            "total_reward:281113.46135500714\n",
            "total_cost:  99.90638327732879\n",
            "total trades:  2250\n",
            "Sharpe:  0.7026063856823678\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080609.6161442325\n",
            "total_reward:980609.6161442325\n",
            "total_cost:  99.8974009731182\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705602895308748\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.5633120066\n",
            "total_reward:980615.5633120066\n",
            "total_cost:  99.89794435650313\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613143272277\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.5633120066\n",
            "total_reward:980615.5633120066\n",
            "total_cost:  99.89794435650313\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613143272277\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.5633120066\n",
            "total_reward:980615.5633120066\n",
            "total_cost:  99.89794435650313\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613143272277\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<stable_baselines.ddpg.ddpg.DDPG at 0x7f19e6eb5c88>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1sve9WGvsC__"
      },
      "source": [
        "### Model 3: A2C"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "QoX5tp7bTnTi",
        "outputId": "dc4ea660-26eb-449f-af68-a15666b3d3a8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "#tensorboard --logdir ./single_stock_tensorboard/\n",
        "env_train = DummyVecEnv([lambda: SingleStockEnv(train)])\n",
        "model_a2c = A2C('MlpPolicy', env_train, tensorboard_log=\"./single_stock_trading_2_tensorboard/\")\n",
        "model_a2c.learn(total_timesteps=100000,tb_log_name=\"run_aapl_a2c\")\n",
        "#model.save('AAPL_a2c_50k')\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/stable_baselines/a2c/a2c.py:184: The name tf.train.RMSPropOptimizer is deprecated. Please use tf.compat.v1.train.RMSPropOptimizer instead.\n",
            "\n",
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/rmsprop.py:119: calling Ones.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Call initializer instance with the dtype argument instead of passing it to the constructor\n",
            "previous_total_asset:100000\n",
            "end_total_asset:143568.45233873036\n",
            "total_reward:43568.45233873036\n",
            "total_cost:  6387.934081530226\n",
            "total trades:  2456\n",
            "Sharpe:  0.4585063869500929\n",
            "previous_total_asset:100000\n",
            "end_total_asset:471519.15020942007\n",
            "total_reward:371519.15020942007\n",
            "total_cost:  3212.7233202738003\n",
            "total trades:  2510\n",
            "Sharpe:  0.8100652990954699\n",
            "previous_total_asset:100000\n",
            "end_total_asset:711997.8256528199\n",
            "total_reward:611997.8256528199\n",
            "total_cost:  1962.9036105102316\n",
            "total trades:  2509\n",
            "Sharpe:  0.9281065879650406\n",
            "previous_total_asset:100000\n",
            "end_total_asset:813730.9830055628\n",
            "total_reward:713730.9830055628\n",
            "total_cost:  5242.129027291222\n",
            "total trades:  2515\n",
            "Sharpe:  0.9784477406469194\n",
            "previous_total_asset:100000\n",
            "end_total_asset:763906.3490314147\n",
            "total_reward:663906.3490314147\n",
            "total_cost:  4817.21023018733\n",
            "total trades:  2513\n",
            "Sharpe:  0.9614375402950348\n",
            "previous_total_asset:100000\n",
            "end_total_asset:666065.8786650962\n",
            "total_reward:566065.8786650962\n",
            "total_cost:  4676.52369838125\n",
            "total trades:  2515\n",
            "Sharpe:  0.9251937168254919\n",
            "previous_total_asset:100000\n",
            "end_total_asset:828059.3046764581\n",
            "total_reward:728059.3046764581\n",
            "total_cost:  2756.847100758158\n",
            "total trades:  2515\n",
            "Sharpe:  0.9794847255543758\n",
            "previous_total_asset:100000\n",
            "end_total_asset:820407.5433832017\n",
            "total_reward:720407.5433832017\n",
            "total_cost:  3434.9401856560908\n",
            "total trades:  2515\n",
            "Sharpe:  0.9770267066894319\n",
            "previous_total_asset:100000\n",
            "end_total_asset:767188.7455583132\n",
            "total_reward:667188.7455583132\n",
            "total_cost:  3297.0731466450547\n",
            "total trades:  2515\n",
            "Sharpe:  0.9538168460285256\n",
            "previous_total_asset:100000\n",
            "end_total_asset:778635.5451063968\n",
            "total_reward:678635.5451063968\n",
            "total_cost:  4326.622281863198\n",
            "total trades:  2514\n",
            "Sharpe:  0.9661988894055867\n",
            "previous_total_asset:100000\n",
            "end_total_asset:271013.7760203545\n",
            "total_reward:171013.77602035447\n",
            "total_cost:  6035.33971987824\n",
            "total trades:  2511\n",
            "Sharpe:  0.6001095053202498\n",
            "previous_total_asset:100000\n",
            "end_total_asset:693888.6297474115\n",
            "total_reward:593888.6297474115\n",
            "total_cost:  6725.654488365585\n",
            "total trades:  2512\n",
            "Sharpe:  0.9477727396021005\n",
            "previous_total_asset:100000\n",
            "end_total_asset:695857.6520784842\n",
            "total_reward:595857.6520784842\n",
            "total_cost:  6559.052958980748\n",
            "total trades:  2515\n",
            "Sharpe:  0.9570767009582097\n",
            "previous_total_asset:100000\n",
            "end_total_asset:665060.478724533\n",
            "total_reward:565060.478724533\n",
            "total_cost:  6428.776309423134\n",
            "total trades:  2513\n",
            "Sharpe:  1.0284509105263875\n",
            "previous_total_asset:100000\n",
            "end_total_asset:304313.7964066868\n",
            "total_reward:204313.79640668677\n",
            "total_cost:  6560.517864200142\n",
            "total trades:  2463\n",
            "Sharpe:  0.685670350587845\n",
            "previous_total_asset:100000\n",
            "end_total_asset:556665.7202818475\n",
            "total_reward:456665.7202818475\n",
            "total_cost:  6514.5848537590755\n",
            "total trades:  2515\n",
            "Sharpe:  0.918155329470793\n",
            "previous_total_asset:100000\n",
            "end_total_asset:786268.58362437\n",
            "total_reward:686268.58362437\n",
            "total_cost:  6518.325190759874\n",
            "total trades:  2515\n",
            "Sharpe:  1.0064220446414565\n",
            "previous_total_asset:100000\n",
            "end_total_asset:821071.183253775\n",
            "total_reward:721071.183253775\n",
            "total_cost:  6503.402347123818\n",
            "total trades:  2515\n",
            "Sharpe:  1.0411410205557572\n",
            "previous_total_asset:100000\n",
            "end_total_asset:925445.9804322764\n",
            "total_reward:825445.9804322764\n",
            "total_cost:  6784.712702513881\n",
            "total trades:  2515\n",
            "Sharpe:  1.0404912926059942\n",
            "previous_total_asset:100000\n",
            "end_total_asset:962517.7903138469\n",
            "total_reward:862517.7903138469\n",
            "total_cost:  5193.491431148093\n",
            "total trades:  2515\n",
            "Sharpe:  1.0444274794914112\n",
            "previous_total_asset:100000\n",
            "end_total_asset:972332.4928350974\n",
            "total_reward:872332.4928350974\n",
            "total_cost:  6897.381129094436\n",
            "total trades:  2515\n",
            "Sharpe:  1.0570876131621412\n",
            "previous_total_asset:100000\n",
            "end_total_asset:945060.4014321667\n",
            "total_reward:845060.4014321667\n",
            "total_cost:  6735.608970433232\n",
            "total trades:  2515\n",
            "Sharpe:  1.0487443970392183\n",
            "previous_total_asset:100000\n",
            "end_total_asset:918670.8822703442\n",
            "total_reward:818670.8822703442\n",
            "total_cost:  6680.795828343729\n",
            "total trades:  2515\n",
            "Sharpe:  1.0403145093565656\n",
            "previous_total_asset:100000\n",
            "end_total_asset:932346.3690885586\n",
            "total_reward:832346.3690885586\n",
            "total_cost:  6617.01394678098\n",
            "total trades:  2515\n",
            "Sharpe:  1.047549236579285\n",
            "previous_total_asset:100000\n",
            "end_total_asset:929525.1868265397\n",
            "total_reward:829525.1868265397\n",
            "total_cost:  7027.402624023227\n",
            "total trades:  2515\n",
            "Sharpe:  1.0488654810492868\n",
            "previous_total_asset:100000\n",
            "end_total_asset:908426.3718980909\n",
            "total_reward:808426.3718980909\n",
            "total_cost:  6743.710549693906\n",
            "total trades:  2515\n",
            "Sharpe:  1.0533517402923307\n",
            "previous_total_asset:100000\n",
            "end_total_asset:837756.2032604109\n",
            "total_reward:737756.2032604109\n",
            "total_cost:  7696.352012277036\n",
            "total trades:  2515\n",
            "Sharpe:  1.0367535626224442\n",
            "previous_total_asset:100000\n",
            "end_total_asset:863230.8268439324\n",
            "total_reward:763230.8268439324\n",
            "total_cost:  7522.8635206924\n",
            "total trades:  2515\n",
            "Sharpe:  1.024741392144145\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1068293.4052346733\n",
            "total_reward:968293.4052346733\n",
            "total_cost:  3950.7904742669434\n",
            "total trades:  2515\n",
            "Sharpe:  1.0696609048755705\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1024396.5737258486\n",
            "total_reward:924396.5737258486\n",
            "total_cost:  7377.394023182965\n",
            "total trades:  2515\n",
            "Sharpe:  1.0612899463024488\n",
            "previous_total_asset:100000\n",
            "end_total_asset:985937.795078817\n",
            "total_reward:885937.795078817\n",
            "total_cost:  7331.481581167244\n",
            "total trades:  2515\n",
            "Sharpe:  1.0531098337914353\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1039037.4584408774\n",
            "total_reward:939037.4584408774\n",
            "total_cost:  7189.234271772797\n",
            "total trades:  2515\n",
            "Sharpe:  1.060455776955667\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1063517.41348354\n",
            "total_reward:963517.4134835401\n",
            "total_cost:  4901.12351607559\n",
            "total trades:  2515\n",
            "Sharpe:  1.0676215370687918\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1031403.3840595726\n",
            "total_reward:931403.3840595726\n",
            "total_cost:  7692.226820285462\n",
            "total trades:  2515\n",
            "Sharpe:  1.0678040027978382\n",
            "previous_total_asset:100000\n",
            "end_total_asset:961527.0898816795\n",
            "total_reward:861527.0898816795\n",
            "total_cost:  7209.059959788929\n",
            "total trades:  2515\n",
            "Sharpe:  1.0537933358617513\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1048865.3337775513\n",
            "total_reward:948865.3337775513\n",
            "total_cost:  3580.3174852377883\n",
            "total trades:  2515\n",
            "Sharpe:  1.0628302218014327\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1072529.5638042036\n",
            "total_reward:972529.5638042036\n",
            "total_cost:  1634.1296899727581\n",
            "total trades:  2515\n",
            "Sharpe:  1.0683705166511808\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1033626.8291941078\n",
            "total_reward:933626.8291941078\n",
            "total_cost:  6125.992164563634\n",
            "total trades:  2515\n",
            "Sharpe:  1.067705030388622\n",
            "previous_total_asset:100000\n",
            "end_total_asset:821092.0955071684\n",
            "total_reward:721092.0955071684\n",
            "total_cost:  5975.377095007534\n",
            "total trades:  2515\n",
            "Sharpe:  1.0242724391117501\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<stable_baselines.a2c.a2c.A2C at 0x7f19e6d38198>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CiBG2ZknsG73"
      },
      "source": [
        "### Model 4: TD3"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "2Y-v2D2wTnTk",
        "outputId": "9e4b7f22-27e2-412a-b127-84094cb5e24b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "#tensorboard --logdir ./single_stock_tensorboard/\n",
        "#DQN<DDPG<TD3\n",
        "env_train = DummyVecEnv([lambda: SingleStockEnv(train)])\n",
        "model_td3 = TD3('MlpPolicy', env_train, tensorboard_log=\"./single_stock_trading_2_tensorboard/\")\n",
        "model_td3.learn(total_timesteps=100000,tb_log_name=\"run_aapl_td3\")\n",
        "#model.save('AAPL_td3_50k')\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "previous_total_asset:100000\n",
            "end_total_asset:778869.7048272111\n",
            "total_reward:678869.7048272111\n",
            "total_cost:  124.57847519610326\n",
            "total trades:  2504\n",
            "Sharpe:  0.9529387392421051\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n",
            "previous_total_asset:100000\n",
            "end_total_asset:1080615.564949955\n",
            "total_reward:980615.5649499551\n",
            "total_cost:  99.89794448471066\n",
            "total trades:  2515\n",
            "Sharpe:  1.0705613146396975\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<stable_baselines.td3.td3.TD3 at 0x7f19e3d1e748>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KE1Xm9N9TnTn"
      },
      "source": [
        "### Testing data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g1KlgP0FTnTn"
      },
      "source": [
        "test = data_clean[(data_clean.datadate>='2019-01-01') ]\n",
        "# the index needs to start from 0\n",
        "test=test.reset_index(drop=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YGf3nPjuTnTq",
        "outputId": "eeb9b0b1-0e31-4bf8-ea89-f3e9e9da5f48",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "test.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>datadate</th>\n",
              "      <th>open</th>\n",
              "      <th>high</th>\n",
              "      <th>low</th>\n",
              "      <th>close</th>\n",
              "      <th>adjcp</th>\n",
              "      <th>volume</th>\n",
              "      <th>macd</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2019-01-02</td>\n",
              "      <td>38.722500</td>\n",
              "      <td>39.712502</td>\n",
              "      <td>38.557499</td>\n",
              "      <td>39.480000</td>\n",
              "      <td>38.629097</td>\n",
              "      <td>148158800</td>\n",
              "      <td>-2.023387</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2019-01-03</td>\n",
              "      <td>35.994999</td>\n",
              "      <td>36.430000</td>\n",
              "      <td>35.500000</td>\n",
              "      <td>35.547501</td>\n",
              "      <td>34.781353</td>\n",
              "      <td>365248800</td>\n",
              "      <td>-2.206829</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2019-01-04</td>\n",
              "      <td>36.132500</td>\n",
              "      <td>37.137501</td>\n",
              "      <td>35.950001</td>\n",
              "      <td>37.064999</td>\n",
              "      <td>36.266144</td>\n",
              "      <td>234428400</td>\n",
              "      <td>-2.206957</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2019-01-07</td>\n",
              "      <td>37.174999</td>\n",
              "      <td>37.207500</td>\n",
              "      <td>36.474998</td>\n",
              "      <td>36.982498</td>\n",
              "      <td>36.185429</td>\n",
              "      <td>219111200</td>\n",
              "      <td>-2.188346</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2019-01-08</td>\n",
              "      <td>37.389999</td>\n",
              "      <td>37.955002</td>\n",
              "      <td>37.130001</td>\n",
              "      <td>37.687500</td>\n",
              "      <td>36.875229</td>\n",
              "      <td>164101200</td>\n",
              "      <td>-2.093799</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    datadate       open       high  ...      adjcp     volume      macd\n",
              "0 2019-01-02  38.722500  39.712502  ...  38.629097  148158800 -2.023387\n",
              "1 2019-01-03  35.994999  36.430000  ...  34.781353  365248800 -2.206829\n",
              "2 2019-01-04  36.132500  37.137501  ...  36.266144  234428400 -2.206957\n",
              "3 2019-01-07  37.174999  37.207500  ...  36.185429  219111200 -2.188346\n",
              "4 2019-01-08  37.389999  37.955002  ...  36.875229  164101200 -2.093799\n",
              "\n",
              "[5 rows x 8 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lDyxaD_4TnTt"
      },
      "source": [
        "### Trading\n",
        "Assume that we have $100,000 initial capital at 2019-01-01. We use the TD3 model to trade AAPL."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tjSOuewjTnTt",
        "outputId": "15e31522-9dbe-4edb-ff53-dcd0663aa043",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "source": [
        "model = model_td3\n",
        "env_test = DummyVecEnv([lambda: SingleStockEnv(test)])\n",
        "obs_test = env_test.reset()\n",
        "print(\"==============Model Prediction===========\")\n",
        "for i in range(len(test.index.unique())):\n",
        "    action, _states = model.predict(obs_test)\n",
        "    obs_test, rewards, dones, info = env_test.step(action)\n",
        "    env_test.render()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "==============Model Prediction===========\n",
            "previous_total_asset:100000\n",
            "end_total_asset:310696.8490824127\n",
            "total_reward:210696.84908241272\n",
            "total_cost:  99.87627464294434\n",
            "total trades:  456\n",
            "Sharpe:  1.864143095949389\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qYXxFzD5TnTw"
      },
      "source": [
        "# Part 5: Backtest Our Strategy\n",
        "For simplicity purposes, in the article, we just calculate the Sharpe ratio and the annual return manually."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s-uT2flUTnTw"
      },
      "source": [
        "def get_DRL_sharpe():\n",
        "    df_total_value=pd.read_csv('account_value.csv',index_col=0)\n",
        "    df_total_value.columns = ['account_value']\n",
        "    df_total_value['daily_return']=df_total_value.pct_change(1)\n",
        "    sharpe = (252**0.5)*df_total_value['daily_return'].mean()/ \\\n",
        "    df_total_value['daily_return'].std()\n",
        "    \n",
        "    annual_return = ((df_total_value['daily_return'].mean()+1)**252-1)*100\n",
        "    print(\"annual return: \", annual_return)\n",
        "    print(\"sharpe ratio: \", sharpe)\n",
        "    return df_total_value"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Tc6z0wFeTnTz"
      },
      "source": [
        "def get_buy_and_hold_sharpe(test):\n",
        "    test['daily_return']=test['adjcp'].pct_change(1)\n",
        "    sharpe = (252**0.5)*test['daily_return'].mean()/ \\\n",
        "    test['daily_return'].std()\n",
        "    annual_return = ((test['daily_return'].mean()+1)**252-1)*100\n",
        "    print(\"annual return: \", annual_return)\n",
        "\n",
        "    print(\"sharpe ratio: \", sharpe)\n",
        "    #return sharpe"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FxGiNeo6TnT1",
        "outputId": "06abf43e-db27-4834-961f-fa7034abad29",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "df_total_value=get_DRL_sharpe()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "annual return:  100.59183078922125\n",
            "sharpe ratio:  1.8641430959493877\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9Yk3cMeVTnT4",
        "outputId": "747b6e0b-135e-40e4-f791-3b3aacd065a0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "get_buy_and_hold_sharpe(test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "annual return:  97.36192520498028\n",
            "sharpe ratio:  1.7753353475308546\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-J6q_kDOTnT8"
      },
      "source": [
        "DRL_cumulative_return = (df_total_value.account_value.pct_change(1)+1).cumprod()-1"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7Epy4PwtTnUA"
      },
      "source": [
        "buy_and_hold_cumulative_return = (test.adjcp.pct_change(1)+1).cumprod()-1"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HzfBZ0xBTnUE"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L1CW-lrHTnUG",
        "outputId": "e0c053a7-e996-4e80-ba31-ccfd9e7c6d9e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 503
        }
      },
      "source": [
        "%matplotlib inline\n",
        "fig, ax = plt.subplots(figsize=(12, 8))\n",
        "\n",
        "plt.plot(test.datadate, DRL_cumulative_return, color='red',label = \"DRL\")\n",
        "plt.plot(test.datadate, buy_and_hold_cumulative_return, label = \"Buy & Hold\")\n",
        "plt.title(\"Cumulative Return for AAPL with Transaction Cost\",size= 18)\n",
        "plt.legend()\n",
        "plt.rc('legend',fontsize=15)\n",
        "plt.rc('xtick', labelsize=15)\n",
        "plt.rc('ytick', labelsize=15)\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsMAAAHmCAYAAACWBYNaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3zb1b3/8dexvFecxHYSZ9lJgJAASSCEsFdKaSmrjFJaKJSWQktberugvS2X3v5uC720QOnilhIogS4oUPYIIaHMBEIgE0ISx1ke8V6S7fP743xly7I85CXFfj8fDz1kne/Qkb6S/NFHn+85xlqLiIiIiMholBDrDoiIiIiIxIqCYREREREZtRQMi4iIiMiopWBYREREREYtBcMiIiIiMmopGBYRERGRUUvBsAhgjPkvY4w1xhQOwb5P8fZ9xWDvW6JjjMk1xtxvjNntHZMVse7TaOM970uHan0ZOGPMdr03ZDRRMCz9ZoxJN8Zcb4xZZYzZb4wJGGP2GWOeMsZcYYxJjHUfh4sxZr4XUBfGui+RGGOWekFF8NJqjCk1xvzLGHPCAPd9/QEU6N8GfAb4PXAZ8P+GuwPGmGu9Y1BjjEnv4za3eNt80MM6V4Qd4zZjTLUx5hVjzOVh6wa//C0c6OMZDF5/zhui/do+XpYO9v3Hs6F6zgebMeZgY8xvjTGbjDH1xphGY8wWY8zdxpijh/i+D6TPNhmAUROsyOAyxswCngQOBl4AfgaUA/nAEuBeYA7wvVj1cZjNB24CVgDbw5atBNKAwPB2KaJrgTogGZgLXA2caYw53Vq7sp/7vB73mJcORgeH2MeAZ621P4lhH64CtgIzgYuA+3pa2ftSebm3zSxjzMnW2pd72ORO4C1csqMQ+DJwnzFmirX2fwbe/QFLA1rD2m7CPQ+PDvJ9PQJ8GNb2K+/6W2HtWwf5vuNdT8/5IUDMZ+QyxlwF/A5oAh4C1gItuP87FwBfNsbMtdZuGKIuHEifbTIACoYlasaYNOAJYAZwgbX2kbBVbvG+sQ/pt/YDhbW2DfdhHg/+Ya0tD94wxrwMPAZ8Fxe0xxVjTBLgs9YO1vM3Edg/SPtqZ4zJstbW9mG9ecBRuOD2W8AX6SUYBs7C9ft0XEDwRaCnYHiVtfYfIfd5L7AZ+L4x5lZrbUtv/RxKg3gs+3Jf64B1oW3GmJ96yx7oaVtjjA9IsdY2DF0P45O1tjnWfTDGLAHuBjYAH7fW7g5bfiPw9Vj0TUYga60uukR1wX0AWeDnUWxjgaUR2q/wlp0S0vZfXtsc4HZgD9AAvAgc4q3zaeBtoBH3zf3qsP0Wevv4rwj3Gdx/YS9tBbif1dcClbiAdgPwfVyAFr5t+GWpt/wU7/YV3u1Dvdu/7Oa5egjwA3khbZNwGZJib9lu3D+K/D4+/0u9+8wNa8/w2jdF2GYJ8BxQ5T32dcA1EY5rpEvhAI77XOCXQAkug3hKyPqnAd/BZfGagS3AF/rw+Ls7RleErPOlkNdUtffYT+jutYwLTl/BZdpX9PE43AnUes/7N7x9zeplm8e9x2u856UeyO7hOb0wwrLV3rJJYc/Hwn68/+/1Xg+pIW3HevvbDySEtH/Ca/9MpM8COt6nXS4Rnu9jcV8C6oEK4I9AZj/6vx3Y3s1ztwT4kfd8B+h4z54B/BX4yHt9VHmvj5Mj7H+Fdx8FuPdyJe7z61ng4LB1U71jsdlbpwp4D/hF2Hqf8V4HxbjXfTkuo3tEN49xAfB3YJ+3/k6vLzP7+JxvJ8JrGjgP+Ld3DOq8v8/t5jleAczG/YJYi3tP/QOY2MfjtAZoA+ZEcWwzcL9SBj8f9gL3A9PD1kvAZX3XeX2r8Y7BPUBSyOuu2882XUbWRZlh6Y8Lveu7h/h+7sN94P4PkAd8G3jWGPMj4FZccPgn3M/OfzDGbLDWvjKI938ELuj+J+7DNQk4E/g5Liv+FW+9R3DB6tVeXzd67RF/drXWbjTGvAVcaoz5rrW2/SdjY0w2cC7wtLW2zGubBryGK224x9vvLFzJw6nGmIXW2up+PsaZ3nWnbKkx5mpcXe3ruLraelyJwe+MMTOttd/1Vr0M97NzOZ3rb8v62R+AZbiA4zbcP589uH/g4J7fNOAPuH921wJLjTEfWmv/3cM+gz+X/xlYRcdr91VwNbm4kp43gR8AWbjj+ZIx5lxr7VNh+1uI+5n2/+g9s4t3HynA53DZ+XpjzIPA/+IyvT/oZpuJuIDyp9baYF3rt4BL6OP7z7vfabifl6v6sk0vluOCx+NxX1DBfTFoA8biArE1XvtpuGP4Ujf7KsO9hsKPS7j5uF+j7gUexH1Busq7z6v7+0Ai+F/c+/z/6AiQwD3ecbjAqgSYjPvy9KIx5lRr7aqw/WTgfml5HXdsi4BvAo8ZYw4Lec//Bnf878d90UkEDsI9b6Guw30BuBsX4M3EPe5/G2OOtNa215IbYz4FPIx7z/4R97qfCHwcOAxX1taX57wTY8xXvf5uAoJlRlcAjxpjvmKtDd/PZFxA/E/cL0/zcJ+Z2bgvFz3dVxFwJO5Xjj6VQHi/Ij2Le13+A/f5cRDuM+IM73OyxFv9h95j+Bfuc64Vd4zOAVJwX4SG4rNN4lWso3FdDrwL7kO5Ospt+pMh/BdgQtqDmbQaYGpIex5eTVlIWyEDzwynhd5/SPufcR+ek3p6HCHLTqFrFvJrXtsnw9a9ymv/dEjbY0ApMCVs3YW4AKfLY4zQh6Xefg8GcnFZqyXAu177V0PWneQ9nw9G2M8d3mOfEdK2nW4yo/087iuAxG7WfwdIDmmfjAuKH+rtOeiuP7j6yDZcljd03wW44HE7nX8JCGaIlkT5HvhMhMf8T2BX6P7Dtvm+17eikLZ3gDd6eE6v9I5xPq5U6VGvPfT9EXyu+5MZnuxt+/9C2pZ7r9Ma4Hsh7WuA9/pwDCK+TkKWtQHHhLU/iQtaosoO03NmeDOQHmGbjAhtE3CB0lNh7Su8fX0vrP27XvvHQ9r2h2/fTZ8j3f+h3mv/tyFt6bhgrRSYHGGb0Kx9T8/5dkLe07gvOXW4wDo7pD0b9+W8FsgJ294CF4ft9zde+yG9PN6zvfXujOK4ftnb5taw9rO89j+HtL0NbOjja2VFX/ugy4F70WgS0h/ZuA+/oXan9T6RPMHsy+PW2p3BRusyqJtxWYBBY61tDN6/MSbZGDPOGJOLyz4k4ILR/gqWQlwe1n457h/kE979jgE+hfuJtMkbGizX68d23D+nHrMsYTbj/lnuAp4HpgPftdb+NmSdC3HZkXtC78+7z3/hHvuSaB5slG633de1/tZa6w/esNbuwpVKDOTYn4srQbg1bN+7cZnI6bhsZ6h3rbUvRHk/V+GOWWi971Jc0H1mN9t8EZcd2xa2zSJjzNxutvkT7hjvw2W6P4nLXn85yv5GFPKcnwZgjEnFlTA8i3tsp3vtObiM7vJBuNvXrLVvhLUtx2VSCwdh/0G/sxFqhK219cG/jTGZxpjxuC+FbwDHRNhPG64kJlTweQh9rVYDc40xh/XUqeD9Gyfbey8GP/dC7//juC9Ct3nHKXw/bT3dTw8+hst232mtrQnZXw3ucWbS9TNht7X2b2FtkZ6DSLK965oe1+rsfNzz/rPQRmvtk7hSt3ONMcGYpxqYPNCRdGTkUDAs/VGD+xl5qH0UdrvSu94WvqK3bPxg3rkxJtEY85/GmC24TGkF7h/Qn71VxvZ339baYMB7rlcagXHDsp0I/CUkKDsE9z69yrvv8MshuAxVX12A+8f2adxPs1m4usVQh3rXL0S4v+e9ZdHcZ7S29LAs/DUB7rgM5NgXedfrIywLts0Ia++pj10YY6bjgsTngJnGmFneiCxbcF8sr4qwzYl4o7UE1/e2eQP3T7/LNp6f4I7x6cAiYLy19gprbV00fe7FcmChMSYLOA73GlruXU4wxiTjfhFJYHCC4e6OOwzu+z7icTXGzDTG/MUYU4k7XuW498Mnifw5sNt2PVEwUn+v97Z/zxiz1RjzR2NMaNAWvP8Fxpgn6Ki9Db4fDw+7/2CQ+U4vjzNa/XmPDOSYBYPgaP7PFOGe98oIy9Z7+8r1bv8A95m+yhizyxizzBhzqfe6lVFINcPSH+8DJxljZlhrI33gRaOn12D48Eu9tZuQv2036/R2n6F+iTtZ8K+4mrFS3M+yRwK3MPAvk/fjgtKLcbV9l+EeQ2gNavAxPUD3tamNUdznStsxmsQ/jTGNwH8bY9ZYa58Ou8/LcfW6kQzlce/p7P2+HPvhEO0IA1fiXi9XE7nG9VPGmDzvV46gYLD7EzpqNEN93hjzfWtt+JB97/Ujax2t5cA1wEm4rPBua+0mrz45HViMyxy30vPIF33V3XGHwT32XY6rMSYTV/+bgTuh9z1cUNoG3EjX+l7oY3+ttY95X4I/CZyMy65ehQvSllhr/d45AytxAeJ/47LB9bjPuNtxWdl4NJBj9r53Hf6LzKCw1r5mjJmJy6Sf6l0uBf7TGHOCl6yQUUTBsPTHw7h/gl+imxN/ItiPOwElXHg2YbAEP8wGcp+X4YLHS0IbvexcuJ6C7+48hcswXU5HMLzJWvtmyDofevtOHqIA50ZcLesvjTHPWXdiT/BknPI+3mdPj324j3t/BAP7uXQ96XFO2DpRM8YYXE3qWiJP8jER+DXudXCbt00WrlzleSKf4HQEbtSDc3Dvx+H2Eu64n44LhoPZ33W41/TpuADjHWvtYJy0F0un40pZvmitvTd0QXCYtoHwAq8HgAe818rPcSdznosbEeJ8XMB7jrW204mIXrlG6DBowcz2fNyvEIMl9D3yYtiyAb9Hwllrtxlj3gGON8bMttZu6mMfzzTG5ER4zc3BfZloH1bS+6XkYe8SeoLgVcAvgqsN7JHIgUJlEtIff8RlJ75jjDk30grGmKO8D5egLcCxJmTGLWPMWFzGbNBZN+brXuA07x9M8D5n4IYH6otWwjIYxpgMug7WD+7kEogc+HXXxwDuzPgTjDGX4n7ivC9snQpc0PxpY8zi8H14NYR5fb3PCH2oxNX8zQY+6zX/DfcP9mbjxpQOv88xXgYwqI7uH/ewHvd+ehz3T++73hnpABhjJuH6uYOB/ey8BFd3/Gdr7T8iXO7C1RJ/MWSbS3CZyN9H2gYXMDWEbTNsvF8X3sPVsy/EC4a9GvuXcJOJzKXvJRI9vYZiLZjhDP8sOIPI9cJ9YozxeXXV7bznL/haCz4f3d3/l3FfpEI9hwv4vu29fsPvM3Qf0Tznz+Oy0V/3vqgF95eF+/Wsjo4SqsHyfe/6L96oKp14z9/1xphgMP4oLqa5IWy9T+AyzI8Ha6a9mutwb3vXoc9JPL8uZRApMyxRs9Y2eMP3PIkbVuc53AdhBW5kh1NxPz/dGrLZXbjsx3JjzJ+BHNwJPTvo+oE+WO4Cfgo8bYx5FJfduQb3E1xfJgT5B/AVY8xfcfWzE3DBR0WEdd/C/Wz6Qy/Yqwe2RTjpJ9x9uFEyfudtH2kigGtxIx2sNMbcj/tnmYDLrp6LK7f4rz48nu7cgQvwf2SMechaW2KMuRb3pWejd7x24I7t4bgvE3PomGnvdeAqY8x/44aVawP+5Z30E4vjHhVr7WZjzC9w2biV3vEODq2WCXzOhgx/1w/BcofwyWlCPYwLYBZba1/3tmkAnummzw3GmKeB84wxkyOdLNVHXzTGRDp5L7RspjvLcTWvwb9D2y+K0N6T14Elxpjv48bStdbav/Rx26H2Cu6L9W1eSUMJLvN6Ge4LweH93G8WsMcY8zjuPV2Kq3u9FncOxL+89Z7GvRb+bIy5y1t2PK60Yish/8e918VVuM+u940xwaHV8nCfyb/EjfoBUTzn1toqY8z3cJnTN0zH1NVX4IZ5/Irt//COEVlrnzduiMffAZuNMaEz0M3Cnf8wEzdcHLgTS7+Am1ymEFdaMgv4Ku5k0tBfMTcaY17H1d/vpmNoTD8Q+hz09NkmI0ksh7LQ5cC+4GoDv4X7Z1GJq6fdhwuSLyNsuCjc0EI7cFnHjbjA8gq6H2KrMGz7QrofLm0FXYdLSsQF5HtwJ0u8jRuyp8v+u2lLx/1ctsPb/gNc1uF0woZK89b/Am5SDj+dJxY4JdL6Idu95y1/vofnOtfrS/BkvuDg/HfQh0Hp6WbSjZDlP/OWfyGk7Xjc0F+ldEz08RJuvOfQCRfyccHcftw/i/DncUDH3VvWZf2ejn0Pz0NPw0l9GReUNOF+Un0eODGafURYd5y3vzW9rBectOJuXEbVAg/3ss1nvfV+EPYcdZl0I8K2wee6u8vv+7CP4PBXW8PaD/La/UQepizS0GoH4bKaNcE+9PZ89/Sa6KXf28NfL73tC1eW8gzuc67We82diPe+6svrkbDPL9y44T/DjfhR4b0/tuNGAzkobNuTcJ+ztbj3/pO4ILC7+1qEy5SWe/stxo3fHTokYk/P+XYiT7pxPm5s7nrv8ipwXjfPcaTtT6GHz8JunvtDcAHxFtyXgibcL5N/ABaErRucdOMj7/VXijvheXrYejfgguVSOiYl+TtwZNh6PX626TJyLsY74CIiIiIio45qhkVERERk1FIwLCIiIiKjloJhERERERm1FAyLiIiIyKilYFhERERERq2YjTOcm5trCwsLY3X3IiIiIjJKrFmzptxaG3GSqpgFw4WFhaxevTpWdy8iIiIio4QxZkd3y1QmISIiIiKjloJhERERERm1FAyLiIiIyKgVs5rhSAKBACUlJTQ1NcW6KyNeamoqU6ZMISkpKdZdEREREYmZuAqGS0pKyMrKorCwEGNMrLszYllrqaiooKSkhKKiolh3R0RERCRm4qpMoqmpifHjxysQHmLGGMaPH68MvIiIiIx6cRUMAwqEh4meZxEREZE4DIZjzefzMX/+fObOncu8efO47bbbaGtrA2DFihWMGTOG+fPnM3v2bL7zne+0b7d06VKuu+66WHVbRERERPohrmqG40FaWhpr164FoLS0lEsvvZSamhpuvvlmAE488USeeOIJGhsbWbBgAeeffz7HH398LLssIiIiIv2kzHAP8vPzufvuu7nrrruw1nZalpaWxvz589m1a1eMeiciIiIiAxW/meHrrwcvQzto5s+H22+PapMZM2bQ2tpKaWlpp/bKyko++OADTjrppMHsoYiIiIgMI2WGo7Rq1SrmzZvH5MmT+fjHP87EiRNj3SURERER6af4zQxHmcEdKh999BE+n4/8/Hw2btzYXjO8bds2Fi9ezMUXX8z8+fNj3U0RERER6QdlhntQVlbGNddcw3XXXddlKLKioiJuuOEGbrnllhj1TkREREQGSsFwmMbGxvah1ZYsWcIZZ5zBTTfdFHHda665hpUrV7J9+3bADa82ZcqU9ktJSckw9lxEREREomXCR0kYLgsXLrSrV6/u1LZx40YOPfTQmPRnNNLzLSIiIqOBMWaNtXZhpGXKDIuIiIjIqKVgWERERORAcN11EHYOkwycgmERERGRA8FvfuOuW1tj248RRsGwiIiIyIGkoiLWPRhRFAyLiIiIHEjCZsWVgVEwLCIiInIgUTA8qBQMh/H5fMyfP5958+Zx5JFH8uqrrw7Kfq21XH311cyZM4fDDz+c1157rdt1TznlFEKHndu+fTuHHXZYj/tfsWIFn/rUpyIuKywspLy8vH8dFxERkfiQkeGuy8pi248RJn6nY46RtLQ01q5dC8Czzz7LjTfeyMsvvzzg/b7yyit88MEHrF+/nqamJmpqaga8TxERERlF0tKgvl6Z4UGmzHAPampqGDt2LNA183rdddexdOlSli9fznnnndfe/vzzz3P++ed32VdycjL79u0jEAiQlpbGhAkT+tWnpqYmrrzySg4//HAWLFjASy+91GWdiooKzjjjDObOncuXvvQlYjWxioiIiAyi5GR3rWB4UMVtZvjmf61nw+7BzZ7OKcjmprPn9rhOcDrmpqYm9uzZw/Lly3tc/9RTT+WrX/0qZWVl5OXlce+99/LFL36xy3oTJkygtraWK664gmXLlmF6GSfwc5/7HGlpaQD4/X4SEtz3lt/85jcYY3jvvffYtGkTZ5xxBlu2bOm07c0338wJJ5zAj3/8Y5588knuueeeHu9LREREDgCNje5aZRKDSpnhMMEyiU2bNvHMM89w+eWX95hZNcZw2WWX8cADD1BVVcVrr73GJz7xiS7rXXjhhaxcuZL09HS+9a1vAfC1r32NJ554IuJ+ly1bxtq1a1m7di1PPfVUe/srr7zC5z//eQBmz57N9OnTuwTDK1eubF/nrLPOas9ui4iIyAGsvt5dKzM8qOI2M9xbBnc4HHvssZSXl1NWVkZiYiJtbW3ty5qamtr/vvLKKzn77LNJTU3loosuIjGx89NaWlpKeXk5RUVF/OEPf+CCCy7g5ptv5q233uLWW28dtscjIiIiByi/311AwfAgU2a4B5s2baK1tZXx48czffp0NmzYQHNzM1VVVbz44ovt6xUUFFBQUMBPf/pTrrzyyi77ycvLw1rLSy+9hM/n4+677+aOO+7gyCOPJCN4ZmgfnXjiiSxbtgyALVu2UFxczCGHHNJpnZNOOokHH3wQgKeffprKyspoH7qIiIjEk2BWGEAjRA2qXjPDxpipwP3ABMACd1tr7whb5xTgMWCb1/SItfYng9vV4RGsGQY3HNp9992Hz+dj6tSpXHzxxRx22GEUFRWxYMGCTtt97nOfo6ysjEMPPbTLPo0xPPzww3zjG9+goaGB9PR07rrrLm699Vb+8Y9/cOGFF/a5f1/96le59tprOfzww0lMTGTp0qWkpKR0Wuemm27is5/9LHPnzuW4445j2rRp/XgmREREJG6EBsMhv07LwJneRhowxkwCJllr3zbGZAFrgPOstRtC1jkF+I61NvJAtxEsXLjQho6lC7Bx48aIweSB4LrrrmPBggVcddVVse5Knx3Iz7eIiMiosmkTHHooZGZCVhbs3h3rHh1QjDFrrLULIy3rNTNsrd0D7PH+rjXGbAQmAxt63HAUOeqoo8jIyOC2226LdVdERERkJKqrc9fjxkFDQ2z7MsJEdQKdMaYQWAC8EWHxscaYd4HduCzx+gH37gCxZs2aWHdBRERERrJgMDx2LFRVxbYvI0yfT6AzxmQCDwPXW2vDBwB+G5hurZ0H/Bp4tJt9XG2MWW2MWV2mMfJERERE+iZYMzxuXMeoEjIo+hQMG2OScIHwMmvtI+HLrbU11to67++ngCRjTG6E9e621i601i7My8uLeF+aLW146HkWERE5gIRmhhUMD6peg2Hjpkq7B9horf1lN+tM9NbDGLPI229FtJ1JTU2loqJCgdoQs9ZSUVFBampqrLsiIiIifVFXR01yOttzp0JbG7S2xrpHI0ZfaoaPBy4D3jPGrPXafgBMA7DW/h64ELjWGNMCNAKX2H5EtFOmTKGkpASVUAy91NRUpkyZEutuiIiISF/U13PJpT9jw9iZbOcOCATA54t1r0aEvowm8QpgelnnLuCugXYmKSmJoqKige5GREREZGSprWXDBDcPQpMviVS/H/QL76DQDHQiIiIi8W7btvY/K9JzVDc8iBQMi4iIiMS79R0j1lZkjHFlEjIoFAyLiIiIxDNrYcMGEm0boMzwYFMwLCIiIhLPdu2CmhrSvKitLEPB8GBSMCwiIiISz7wSidRkN3pERbrKJAaTgmERERGReOYFw9bnBgErz9DEG4NJwbCIiIhIPNuwAfLzaWgJ1gyPUTA8iBQMi4iIiMSz9etpmzOHBr+bda5cNcODSsGwiIiISLzyRpJonHtEe1N5eo5qhgeRgmERERGReFJTA7NnwyuvQEkJ1NRQP3tO++L65DRlhgeRgmERERGRePL++7B5M7zwgqsXBupnHAxAcgIEfIkKhgdRYqw7ICIiIiKe1avhzTfd35s2QXY2APXTZwDvk5PiI5CgYHgwKRgWERERiQfbt8Oxx0JLi7u9aRNkZLiRJNKzAMhJSWCPL1E1w4NIwbCIiIhIPPjFLzoCYXClEsnJMHcu9X7XnpOaSLEvEfz1MerkyKOaYREREZFY27sX7rkH0tI62pqa4K23YM4cGprdsGo56UkEfEkqkxhECoZFREREYu32213pw623dl0WmhlOS6Y1wUdrs4LhwaJgWERERCSWKivht7+Fiy6Cz3/etf3wh3DGGe7vo4+modkLhjOSAQgEWiLtSfpBNcMiIiIisfSb30BtLdx4I+TkQEUFjBkDxkBxMRQWUv/ShwCMyUgBoMUfRyfQrVrlTv677LJY96RflBkWERERiaXnn4dFi2DePHd73Djw+SAhAQoLASiuaGBcRjKZ6S4YDsRTMPyrX7lA/gClYFhEREQklhoaIDe3x1U276vlkAlZJCW7H/UDgdbh6Fnf7N0L+/fHuhf9pmBYREREJJYaGiA9vdvFbW2WD/bVcsjELJJ8LnTz++OoZnjPHmhsdJcDkIJhERERkVjqJRjeVdVIvb+1UzAcNyfQWesyw3DAZocVDIuIiIjEUi/B8Oa9tQAcPCEOg+GaGjceMigYFhEREZF+6C0Y3hcMhjNJ8hkA/C1tw9K1Xu3Z0/G3gmERERERiYq1fcoMT85JIys1iaRELzPcEicn0AVLJMANCXcAUjAsIiIiEit+P7S19RgMb/FOngNIDpZJKDM8aBQMi4iIiMRKQ4O7DguGN+2toaW1jUBrG1vL6jh4gguG22uG4zEzrGBYRERERKISIRjeXdXIJ+9YxaNrd7OtvJ5Aq+WQiZkAHTXDrXbYuxrR3r2QnAwpKQdsMKzpmEVERERiJUIwvGVfLW0W1u+uJsWrET5kQjYQkhlujaMyiYkTIRA4YGuGFQyLiIiIxEowGE5Lo63N8sNH36OsthmAD0vryExJxJdgmJGXAUByYpwFw3v3wqRJUFd3wGaGVSYhIiIiEivBWdvS03luw14eenMnL2wsBWBraR2b9tZSOD6d1CQfEJIZbvJDSxyMNbx3r8sMZ2TAI4/ApZfGukdRUzAsIiIiEiteZtimpXHHix92WrS7uol3iiuZPTG7va29Znj7DrjqquHrZ3eCZRJXXuluP/QQvPNObPsUJa2igCgAACAASURBVAXDIiIiIrHiBcMvVPvYuKeGY4rGdVpcXudvH0kCQoZWS0iE1auHr5+RBAJQXu7KJK65BiorISsLfvvb2PYrSgqGRURERGKloQEL3Lm5kWnj0rn53LkATBvXcUJdcCQJCCmTOPW0zmP8xsK+fe564kR3nZMDRx0FGzbErk/9oBPoRERERGKloYGXZizkvQo/t14wm0MmZHHjJ2Zz5mET8be08eCbxZx4UF776u0z0OWMdZnYujrIzOxu70PLG2O4Nm8SjbVN5GelwsyZ8MQTselPPykzLCIiIhIrDQ0sW/BJCrKSOf/IyRhj+MrJM5k+PoODJmRx09lzyUjpyF0mJria4cCYsa5h585Y9NrxguGflWXwhT+95dpmzXIZ49ra2PUrSgqGRURERGKloYGK9Gxm5WW0l0D0pL1MInuMayguHsre9cwr09jm91FS6Q0RN2uWu966NUadip6CYREREZFYaWigKTGFtNTkPq3uSzD4EgyBLG+EiTjIDO9rttQ2tdDS2ubKJEDBsIiIiIj0QUMDDclppCX7+rxJks8QSMuAhITYZob37oVx4yir9QNQ1RjoCIY//LCHDeOLgmERERGRWGlooDE5lbTkvo9pkORLwG+BgoKYl0k0Tp5KbbOb/KOqwQ/Z2a5f69bFrl9RUjAsIiIiEivBMomkvmeGk30JbjrmadNiXiZROnVm+83KhoD747jj4N//jlGnoqdgWERERCRGbGMjDYnJpCX3PSRL8iUQaLEwdWrMM8P7Jkxrv1lZ78olOOEE2LEDSkpi1LHoKBgWERERiZFAdQ2tCb6oMsNJiaZzZtjaIexhN6x1meHcSe1NVcHM8PHHu+sDJDusYFhEREQkRhrLKwGirxkOBsPNzVBWNlTd615NDTQ1UZqV295U2eBlhufPh4wMBcMiIiIi0rOmCi8Y7k/N8NSpriEWpRLeGMOl6WNI8hmSfQkdNcOJiXDMMfDKKy5Y//nP3Ux5cUrBsIiIiEgsWEtDtQsSo64ZbrUuMwyuVKK+Hs4+Gz76aCh62pU3xnBpYgb5WamMSU9yo0kEnXACvPsu3Hcf3Hiju8QpBcMiIiIisVBZSSMuIxxVzbAvpGYYXGZ40yZ44glYtWooetpVMBgmibysFMamJ1FW2+wm3gBXN9zWBg8/7G4/+eTw9KsfFAyLiIiIxMLevTQmpQDR1wzv3N9AbXoWpKW5YLimxi2srh68/t1+O/z1r5GX7d4NQGnAkJ+VQkZKIi9uKuXqP69xyxcvdpOCPP+8u71tG2zZMnh9G0QKhkVERERiYd8+moLBcBSZ4QRj2F7RwA2PvN8xokRtrVtYVTU4fWtpgW99Cy65xN1ua4NbboF9+9zt4mLIzKS0voUJ2alccVwhqUkJLN9Uypvb9rvJN444wo06keudZPfqq4PTt0GmYFhEREQkFvbupaEfwfC3PnYwABv31HSMNRzMDA9WMPzuu51vv/023HBDR9lDcTFNRTOpagyQn5XCufMn886PzmBcRjIPvemd0HfCCe76nHMgKwveemtw+jbIFAyLiIiIxMK+fTQmpQKQltz3YHhR0TiuPL6QfTVN2PDM8GCVSaxc6a4zM9316tXuev9+d11cTFnRIQDkZwdLPXzMLchma5k3ckQwGJ4zB44+Gt58c3D6NsgUDIuIiIjEwo4dNKWmA9EFwwCTc9Ko97dSPbXIDXNWUeEWDFZm+JVX3HVGhruOEAyXTpkBQH5WavtmRbkZbCuvx1oLp58OCxbAkiWwaJHLNjc1DU7/BpGCYREREZHhtmwZ3H47DbNcdjWaMgmAKWPTACjJn+bqcjdtcgtCg+FAAH7wA/j+96Pv38aNnfcXGgw3NEBZGaX5U4COzDBA4fgMapta2F/vd7XCb78N8+bBscfC7Nnt4xPHEwXDIiIiIsNt2TKYNYvGL30FiD4YLshxwfDunAmuYf16dx0aDP/85/Czn8Gtt0bXt7a2jvGKm5uhshLef9/d3r/flWUApTn5QOfMcGGuy3Rvr6jvvM9zzoF166CoKLq+DAMFwyIiIiJDqakJLrsMfvlLF6y2tLgyhCVLaDQuCE5Nii4km+wFw7vSclxDMJMbWjO8fLm7NsYFuH2xbh386lcuCD7iCNf28svQ2uqGSnv1VZfhBUrTc/AlGMZnJLdvXjjelVVsK2+I6vHEkoJhERERkaH07rvwwAPw7W/D5Mlw8cXuhLeTT6Yp0Epakg9jTFS7HJeRTGpSAruMC4ppbnbXwcxwayus8cb8tbZjtIne/PCH8J3vuL+POspdv/CCu160qKM2OTeX0owccjOTSUjo6PvUcen4EgzbyuN3+uVwCoZFREREhlKxN9TY/fe7cXufegp8Pjj5ZBr8LVGfPAdgjKEgJ41ddYGOcXzBBcPWwubNLuA+7TTXHjzxrSeBgMsCBy1c6K5feAEmTIDDD+9YtnMn+5osE7JTO+0iyZfAwROyWLtzkE7kGwYKhkVERESGUjAYPvtsuOceKClxJ6RNmkSjvy3qeuGgyTlp7K5qdGMNB7W2Qn19xzBmZ57prvsSDL/1VscQbdAR/G7e7ALj8ePd7YkTITWV0pom8rNSuuzmmKJxvL2jyk0ZfQBQMCwiIiIylIqL3aQTY8a427m5MH8+AI2B/mWGwY0osauq0c1CF6q62gXD2dlwzDGurbKy9x2++KKrLw4KzTgvXAjjxnVqL6ttJi+rc2YY3DjIjYFW3t81iFNDDyEFwyIiIiKDYfVq+NvfurYXF7uANUJdcEWdn3EhJ6BFY3JOGuV1fpqmFbqGdDeSA1VVLhheuLAjoO1LZviFF9y4wLt3u2xwTk7HstBgODubQGsbFfV+JmR3zQwfXejWe3NbH+4zDigYFhERERkMt98O113XtT0YDEfgsqtdA8q+aB9ercAbrixYLrF3rztpb9EiGDvWtfUWDNfXw2uvuQkyJk2Cgw/uHAwfdVTH7awsymrdCXv5ETLDeVkpzMjNaA+GqxsD/Xp8w0XBsIiIiMhgqKyE8nI3dFqo4mKYPj3iJmW1zeRl9i8Ybh9ebfxkAJ5f+HH8CYnw+OOuD6HBcG9lEqtWuRPolizpaEtNheRkNwLGpEkdI1ZkZVHaHgxH7vuionG8tX0/7xRXMu/m53jm/fibbCNIwbCIiIjIYKiudiM5lJd3tDU0uNuhJ7l5Gv2t1Da3dJrBLRqTvVnodmWO58PxU/jylDN44ZBj4cEH3QqLFrmANj2998zwCy+4wPf44zvajHHBdHCIteCyr3+d0ho3rXJ3fV9UNI6aphb+troEgN+t2NqvxzgcEmPdAREREZERITjG7969biiyb36zfYIKJkzosnqw1KC/meEJ2akkGNidOobKdHdyXsXUGbBxFRQUuIwuuIC2t8zwiy+6YDdYdwys311N+u1/oGiOV4YxfboL9oF9r+8AIpdJQEfd8BPrdnv7qqGqwU9Oev/qo4dSr5lhY8xUY8xLxpgNxpj1xphvRljHGGPuNMZ8aIxZZ4w5cmi6KyIiIhKngsHwvn2u/vbXv4Y//cm1hdbfesrqXHa1vzXDSb4EJmanUhJIoO63dwNQU+DVJh99dMeK48b1nBkuK4O1a+H009ubrLWcdecrnLo2sX0muvd3VeNvccOl7a5qJMlnuu37lLFpFIxJpbbJlYy0tFneiNMT6vpSJtECfNtaOwdYDHzNGDMnbJ1PAAd5l6uB3w1qL0VERETiXWgwfPvt7u+tXnlAsHY3RE8nofXV5LFp7KpspHZqIQA1+QVuwaJFHSv1FgyvWuWugxN0ADv3N3ZaZUdFPWff9QpPvedqf0sqG5k0Jg1fQuSZ84wxLCpy2eGPz51Aks/E7UQcvQbD1to91tq3vb9rgY3A5LDVzgXut87rQI4xZtKg91ZEREQkHrW0uBEZAN54Ax5+2P0dDJAjBMPBk9D6mxkGb+KN6kbqvAxsTY43lFpoZjg31wXo4crLXdnD3r3u9owZ7Yte/8hNuxwcDe7t4kqshT3VLpu9q7Kh/QS+7iwqcpN0zC0Yw5xJ2bxT3IexjmMgqhPojDGFwALgjbBFk4GdIbdL6Bowi4iIiIxM1SETTPzxjy6KDA1II5VJ1DaTYOj3OMPghlfbU9XUPnxZTcFUuO02OPXUjpWKimDHDmgLmRFu40bIy4P/+7+OvgcnBQFe3+aC4eBoEe/udOtUNfgB2FXV2H4CX3dOmJVLsi+Bo6aPZf7UHNaVVNMSh7PS9TkYNsZkAg8D11tra/pzZ8aYq40xq40xq8vKyvqzCxEREZH4UxVSAuD3wwUXtM8yB3TJDK/6oIw/rtrGzLzMbksN+mLy2DRa2iwfldUBUNMC/Md/QGIi1lr+9tZOHsg7ApqaOjLA0DFd88svu2A4OdmNPOHZsNuFejWNLuMcLHGobPDjb2mjtLa518zwtPHpvPPjj3H8rFwWTBtLS6uleH9Dvx/rUOlTMGyMScIFwsustY9EWGUXEDpmyBSvrRNr7d3W2oXW2oV5eXn96a+IiIhI/KkOm3r4+us7AmBj3NTIIf782g6y0xL50xVHMxDBgHTzvlrXjZAJLpa+up3vPbyO/9zvzRz30UcdGwbLJiZMcH0P6Z+1lh0VLmhtDLRS39zSHhxXNQTYU92Ite4kud5kpLiBy848bCLv3XwGM/Iy+/dAh1BfRpMwwD3ARmvtL7tZ7XHgcm9UicVAtbU2fkdXFhERERlMwczw/Plu4orFi9unLw6MHcfNT27kb6t30uhvpc0bWeHkg/OYOi69h532LhgMf7DPywyHBMPLN5UCYIA2TOdgOJglzshwwbBXIrGtvJ6/rymhMdDKrHwXuL65bT9+r7xhXUk1J/9ihbvvPgTDQalJPlISfVE/vuHQl3GGjwcuA94zxqz12n4ATAOw1v4eeAr4JPAh0ABcOfhdFREREYlTwWD4vvvg8MNdNtgLhjcUzuXef28H4KdPbOC02flUNwY4bmbugO82GJA2BloBqPFOpGtrs6wtdn2yQF1KOtnbtnVsGAyM6+qgpqY9GP6fpzby/AaXNT5iyhg+LK1j5Qdl7bfXlXRkwA+d2DnbfaDqNRi21r6C+1LR0zoW+NpgdUpERETkgBIMhseM6RiCwQuGS/Ld2L//fd5hvPFRBU++t4cEA8fOHD/gu01PTmRsehKVDS4jXN0YwFrL1rI6aptbOKZoHG9s209V0cFkh2aGg0O+1da2Z4YDrW28trWifZV5U3J45O1drNxSRm5mMnMLstuD4U3/fSapSfGZ6Y2WZqATERERGahgzXDoqBFeMLxrrBtt9px5BVy2eDpltc3sqW5kQnb/xxcOVZCT1h4Mt7ZZGvytvOOd8Hba7HwXDBfOZFpxsdvA2o7McDAYnjmTtTurqGtuad/voZNc5ndrWT2nz85vnz0uPytlxATCEOXQaiIiIiISQVWVywhnZXW0BYPh7DyyUhIZk5YEuHGFj5jSdai1/gof1aGmKcA7xVVkpyayYJo7ia9qbF7HlMzFxdDgjeoQDIazs1m1pYzQgS3GZSS1/z1vag5j05O89vibUnkglBkWERERGYhAAP7xD1Yd/yk+eHUHXzyhyLUHg+G0sVGdbBat4L4zkn3U+1upaWzhneJK5k8b2x7QVmaP75iF7g1vuojMzE5lEis/KGfe1Bz+9IWjaQy0kujriIznTc1hnzfhRmbKyAoflRkWERERGYh774UNG/jbOV/mzuUfdLQHa4aTs/o0DFl/BTPDwaB4d1UjW/bVMn9qTntpQ3VGTkdm+I033JjCxx/vTp6rqaEqexzrSqo48aA8xmYkU5CTRnZqSGZ4yhiyUl0QnJmqYFhEREREgpYtgzlzqBk/gaqGAM0tbmQH0tPhoovYlZjZ6wQVAxHc92xvdId/vbubNgsLpuW0l2ZUpWe5kSMCAXj9dTjqKBes790L1vJq2iTaLJx0UMcIF244tAQKx6eTk55Mk/e4lBkWEREREWffPli1Ci68sH3Ci4o6N2WxBWrvX0ZtqzvJbagEM8KHTsomOTGBp953Uz3Mn5JDki+BzJREqlK8yS5KS2HNGjcOclaWuw2sYixZKYnMm9q5ljk/O4Ujp7u64yWHTuD02fl8/8zZQ/ZYYmFkhfYiIiIiw+lf/3KjM3z609S84MoQymqbyctK4exfv8KiIlcqMT4zZci6UJibQVZqIgflZzIzL5ONe2qYkZvBWO9Et5z0JKqsF4yvWAHNzS4Yfv11wAXtK5vTOHbWeJJ8nfOk915xNGO9Uous1CTuGeCMefFImWERERGR/nr+eSgogCOOoKbRDUtWVtvMU+/tYdPeWl71xu0NlisMhezUJN750cc4/dB8DvJmjZs/rSPDm5OeRFWCNwLE00+762OOaR/5YtvYAnb5Ezjx4Lwu+56VnzWkgXw8UGZYREREpD/a2mD5cvjEJ7B0TIVcWtvMA6/vAKB4vxvCbCiDYYBEL6MbDIYXhJQ75KQlU9ngjQv87LMueJ8ypT0YXltwCADHzhg3pH2MV8oMi4iIiPTHunVQXg5LltDc0oa/tQ2AR9fuYsOeGhITDP4W1zbUwXDQEVNzMAaOmdExu11+dgp7m70b5eWweDEWuK9tAtUpGVSluqA4d4RngLujYFhERESkP154wV2ffnp7VhjgzW37yc1MYcmhE9rbhisYPumgXF694TQOntAx+cf0cRnsbWilyef1YfFiVmwp46byHG49+QvUpGYAI2+UiL5SMCwiIiLSHy++CLNnw+TJ7SNJBF1x3HQmZHdkWocrGDbGMGlM55Erpo1PwwIlOV5wvngxJV75RkuCj9qCaWQk+9pLLUab0fmoRURERAbC74eVK2HJEsBNgQzgSzCkJfn43DHTGeONwpCcmEBqUuxCrmnjXOZ3x8Qi8PngqKMo84Z/G99QTU1+AVmpwxOsx6PRmQ8XERERGYjXX4eGho5g2BtJ4oYzZzN9fDpjM5IZm+4CzDFpSRhjut3VUJs+Ph3wguGUekhPZ09VIwCtR8yjdt5CsmoCPe1iRFMwLCIiIhKtF16AhAQ4+WSA9jKJ0w/NZ0aeG9EhOD7vcJVIdGd8RjIZyT6Kjz4RZp0EwI4KVyZRf/rHqC2vJzvGfYwlBcMiIiIi0XrhBTj6aMhxQ5gFyyRCg8oxIZnhWDLGMG18BsVj5oE3aca2inoA6ptbqWlsYXxmciy7GFOqGRYRERGJRk0NvPlme4kEdIwxnB1SexsvmWGAvKwUKupdnXBdcwtltW6stdqmFmqbAp36PdooGBYRERGJxrZt0NoK8+e3N63fXUNWSiLJiR2hVU5afGSGwfWlusEFw+t2VrW31ze3UNPUQlbq6C0WUDAsIiIiEg2/CypJTQXgta0VPP3+Xq48vrDTavGUGc5JT6LKy16v2VEJwJHTcqhrdpnh0TyahIJhERERkWgEg+FkF+w+8MYOxmck89VTZ3VaLSs1kdzMFIpyM4a7h13kpCVR3Rigrc2ypriSgydkMiknjYq6ZgKtluy00ZsZHr2PXERERKQ/QoLhBn8LyzeWcsFRk0lN8nVaLSHB8PJ3T+nSHgtj0pOx1o168faOSs46YhJtbbCnpglAmWERERER6aOQYHjF5jIaA62cdXhBxFUzUhLxJcRujOGgYKnG28WV1DS1cOS0sWSmJmKtW56tmmERERER6ZNOwXAp2amJLCoaF9s+9SJ4Mt+Lm0oBOGr6WDJSOgJgjSYhIiIiIn3jBcM2KYlVH5RzwkG5cZH97UmON+bxS5tKGZueRFFuBpkpHeUbGk1CRERERPrGC4a3Nhr2VDdx4kF5Me5Q74LB8J7qJo6cNhZjDJkpHdngmd6seaORgmERERGRaHjB8Mv73PWJB+XGsjd9MiatY4a5I6ePBSAjJDM8NkMz0ImIiIhIX3jB8KrdDczIy2DK2PQYd6h3oWMdH+UFw5lezfBB+aM3KwwKhkVERESi4/fT7Evk9V31nHQAlEgAJCcmkJ7sw5dgmDclB4CmQBsAh00eE8uuxdzorZYWERER6Q+/nzWT59DU0nZAlEgE5aQlkZuVQlqyK484bXY+nztmGt8+45AY9yy2FAyLiIiIRMPvZ2XRApISDItnjI91b/rsC8cVMiknrf12WrKP/3f+4THsUXxQMCwiIiISDb+fVYULOHJaTqexeuPdV06eGesuxCXVDIuIiIhEoaK5jfUTZ3HSwQdGvbD0TMGwiIiISBT2BtwEGzNH+SgMI4WCYREREZEoBAKtgBuhQQ58OooiIiIiUfC3eMGwz9fLmnIgUDAsIiIiEoVgZjjJZ2LcExkMCoZFREREouBvdZNVJKlMYkTQURQRERGJQqDFBcPJPoVRI4GOooiIiEgUAl5mWCfQjQw6iiIiIiJR8LdaAJKUGR4RdBRFREREohBoD4Z1At1IoGBYREREJAr+NhcMq2Z4ZNBRFBEREYlCIBgMq2Z4RNBRFBEREYlCwJ0/p5rhEUJHUURERCQKfpcYVjA8QugoioiIiETB354Z1gl0I4GCYREREZEoBKwh2bZijILhkUDBsIiIiAjAnj1www0QCPS4WgBIsm3D0ycZcgqGRURERADuuANuuQXefLPH1fwkkIQdpk7JUFMwLCIiIqNTdTW0tLi/29rgoYfc3++91+NmAQxJRsHwSKFgWEREREaf+nrIyYHvftfdfu01KC52f7//fo+b+kkgWZnhEUPBsIiIiIw+jz/urn//e3f90EOQmgqHHw4rVsDKld1uGjAJJCszPGIoGBYREZHR58EH3XVGhiuV+Nvf4OyzYd48WL8eTj65o4QilLUEjI8kDSQxYigYFhERkdHFWnjpJfd3RQXcdx+UlcFnPwunndaxXl1d120DAfy+JAXDI4iCYRERERld9uxxNcOf/7y7/R//AWPGwCc+AV/4ghtRAqC2tuu2fj9+XyJJiqBGDB1KERERGV22bHHXF1/sTqKrqYHzznM1wwkJUFTkltfUdN3W7yeQkEiyIqgRQ4dSRERERj5robnZ/b15s7uePx/+/W845xz49rc71s3KcteRguG6OgK+RJJ9CqFGisRYd0BERERkSBUXuyzwBx/AL37hguG0NJg82WWCH3us8/rZ2e46UpnEu+8S8CWSnZUx9P2WYaFgWEREREYua10d8IYNMHcuXHUVJCbCnDkuEI4kGAxHygyvXo0/MZ+ksTlD12cZVsrxi4iIyMj11FNu3OBbbnElEX/8oyuDWLy4+216KpN46y38aekkpyQNSXdl+CkYFhERkZHr1VfB54Mvfcllgq+6Cvbuhbvu6n6b7sokrIXVqwmkZahmeATRkRQREZGRq6QECgogKSSTm5zc+bZn7c4qzrpzFdWJqa4hNDP8P/8D994LZWUEkpJJUjA8YqhmWEREREaukhKYMqVPqy799zbW767h7d21nJqW1hEMl5bCj34E48YBEPAlkpSoWTdGCn2tERERkZFr165ug+HH391NcUUDAPXNLTy7fh8Am/bUulKJYJnEY49BWxuUlwPgJ4Fkn2/o+y7Dotdg2BjzJ2NMqTHm/W6Wn2KMqTbGrPUuPx78boqIiIhEyVqXGZ48ucui4ooGvvHQO1zzwBoAnl2/l8ZAK0k+w6a9NS4YDmaGH3mk07Z+a5QZHkH6khleCpzZyzqrrLXzvctPBt4tERERkQGqrnbTLkfIDP/znV0A7K1par89ZWwaJx6Ux2Nrd/ONY6/E1tRAVRW8+CJMm+Y2nDiRQGubTqAbQXo9ktbalcD+YeiLiIiIyODw++HOO93fEYLhx991wXBtU4Cd+xv494flnL9gMoXj3WQaj086gua6BnjiCQgE4HvfA6C1aAZtFp1AN4IM1pE81hjzrjHmaWPM3EHap4iIiEj//PrXcNNN7u+wYLjR38pH5fUcOimbQKvllmc20Wbh/AWTuWTR1Pb16hr98PDDrszi858HIDBjFgDJiQqGR4rBOJJvA9OttfOAXwOPdreiMeZqY8xqY8zqsrKyQbhrERERkTDWwtKlHbdnzeq0eFt5PdbCefMLAHhi3R6OmDKGGXmZHDwhi199Zh4Atftr4Jln4NOfhjFj4LLLaD7nXECZ4ZFkwEfSWltjra3z/n4KSDLG5Haz7t3W2oXW2oV5eXkDvWsRERGRrt56C95/H/7wB9i3DyZM6LT4o/I6AE48KI8vnVAEwMULOzLCmd7scnW1DdDU5IJhgPvvJ/DJswBI9ukEupFiwOMMG2MmAvustdYYswgXYFcMuGciIiIi/fHHP0J6OlxyCWRn09ZmWfZmMVtL67jutFlsLa3HGCjKzeA/PzWHy46dzrRx6e2bZ6W68Kg2xWs78cT2ZTv3u6HY8rJSh+/xyJDqNRg2xjwEnALkGmNKgJuAJABr7e+BC4FrjTEtQCNwibXWDlmPRURERLpTXw9/+QtcdFH7tMq/eG4zv1uxFYBH3i4hNyuFgjFppCW7sYKneyfNBWWmuPCo7oG/QL7PTefsWbuzCoD5U3OG/KHI8Og1GLbWfraX5XcBPUzwLSIiIjJM/v53N1nGVVcB8GFpLf+38iMuOHIK154ygx8/tp5Xt1Zw8sHdl2sGM8N1GdlwSOeT79burGJCdgoTxygzPFJoOmYRERGJf9bCpk1w6KE9r3fPPXDwwXDCCQA8/d5eWtosN35yNrmZKSz70jG8vKWMKWPTu91Fe2a4uaXLsnd3VikrPMLoVEgRERGJf08+CXPmuIC4O5s3wyuvuKywcSe47a5uJDczmdzMFACMMZxySD6z8jO73U1msGa4qSMYXrmljLU7q9he0cDhk8cMwgOSeKHMsIiIiMS/DRvc9T//CWlpcP31HcsefhhSU2HlSlffe/nl7Yt2VzVFXdKQkugj2ZfQHgw3+Fv4yp/XkJ/tAuqi3O4DaTnwKBgWERGR+Ld9u7v+wQ/c9RVXQE4O1NXBhRe6tsRE+NSnImNN7AAAIABJREFUYOLE9s32VDd2OUGuLzJTE6lrDgDw0qYyGgOt7KhwI0lMH999iYUceBQMi4iISPzbsaPz7bIyFww/8YS7fc45bqa4r32t02p7qpo4dsb4qO8uKzWROi8z/OR7uzstm6ZgeERRMCwiIiLxL5gZDiorg4MOgr/+FQoKXPlEQudToWqbAtQ2tzApJy3qu8tMSaSuuYUGfwvLN5UyPiOZino/Y9OTyE5NGsADkXijE+hEREQkvlnbNTNcWgo1NfD0025M4bBAuKW1jb++tROASf0YBi0zJZHaJhcINwXauPqkGQBM60fJhcQ3BcMiIiIS3yoq3GQakyd3tJWVweOPQ3MzXHxxl03uf20HP31yIwAF/cgMZ6W6YPjJdXvIy0rhM0e76Zqnj1OJxEijMgkRERGJb8ESiZtuctng73zHZYbfeAOmToXFizutbq3lL28Vt9/uzwlvmSmJlNY2sbWsjkuOnkpOejLnzCvg43Mn9r6xHFAUDIuIiEh8e/lld33GGTB9Otx8M3zwATzzDHz9611KJNbsqGTLvjp+9unDWXLoBPKyUqK+y5z0ZMrr/ACcdUQBAHd+dsHAHofEJZVJiIiISHx79FGYN88FwgD5+W7a5UAAPvOZLqs/+EYxmSmJnDOvoF+BMMAVxxUCMCYtiYXTx/a353IAUGZYRERE4ldZGbz6Kvznf3a05eXB1q1QWAhHH91p9aoGP0+8t4eLjppCRkr/w5zC3Aze/fEZNARaSEgw/d6PxD8FwyIiIhK/nngC2trg3HM72vLz3fXFF7dPuxz0yNu78Le0cekx0wZ812PSkxiDhlEb6VQmISIiIvHrscdg6lSaDjuCh9eUYK3tHAyHsNby4JvFzJuaw9yCMTHorByIFAyLiIjI8Fu3Do4/3k2nHKq11V0AGhrguefgnHN4ev1evv33d3m3pBo++Um45BI48shOm761vZIPS+v43KKBZ4Vl9FCZhIiIiAy/b37T1QK/9hp87GMd7eecA8uXw8knw7Rp0NgI551HcUUjAL9fsZXa5kn8edmDJHglEv/1+HoA9lQ3kpWSyKfmTRr2hyMHLgXDIiIiMvxqa911cnLn9ueeg5YWN4bws8/CmDFw8snsfHQDAM+s3wvA/gY/uZkp7KpqZOmr29s3/8Kx00lPVngjfadXi4iIiAy/YDAcWibR0uJOlvvRj2D+fLjgAlcSkZTEzv0NnTavqHPB8L/e3Q3A3ZcdRaLPcEzR+OF6BDJCKBgWERGR4RcMhmtqOtr27XPBcEEBnHcefPvbcOmlAJRUNnbavKKuGcji8bW7mT81hzM0M5z0k4JhERERGX7BYLi6uqNt1y53PXmym1Xuf/8XgEBrG3uqG0kw0GbdKuX1fj7YV8uGPTXcdPacYey4jDQaTUJERESGl7Ud5RGhmeHdruSBgoJOq++tbqLNwimH5Le3ldc28/i7u0kwcNYROmFO+k/BsIiIiAyv0GxwaDAcmhkOsXmvyyJffdIMNvzk4/gSDOV1Lhg+bmYu+VmpQ91jGcEUDIuIiMjwKin5/+zdd3hUZfbA8e+dyaRMkknvCSEJJXSQXgRhFUEExYqi2LCXtWJd29pW12XtfZVVfyArioBSLEiR3nsJISG9957M/f3xZmYSkgBCCiTn8zw8M3Pvnck7CcrJmfOe47h/fGbYaHQM1aj1874MPF2cGNDJG7OzE77uzvx2IJPEnFKm9K+fRRbiz5JgWAghhBCty1YOAQ1rhkNCVL1wreoaKz/vz2Bcj0BcnIwA+Hu4cCC9CGcnAxN6y8Y5cWYkGBZCCCFE60pXvYJxc3Nkhisq1BCOyMh6l25OyCO3pJIJdbpF+Huo3sRjuwdgcTW1ypJF+yXdJIQQQgjRujIy1G3Xro5g+Lnn4PBheOutepcu35uOi5OBMd0D7Mf83FUwfFn/+rXFQpwOyQwLIYQQonVlZFDk5cuXfcajFxbChg3wxhtw220wcaL9MqtVZ9medMZ0C6g3VS7K3wNfd2fGxQY29upC/CmSGRZCCCFE68rI4NkJ9/F9+BBi921m8E03QXg4/OtfvLcyjqPZJXQP8iTA04X0wnJm9e5e7+l3XxDDjcMjcTUZ2+gNiPZEgmEhhBBCtK6MDOK7nA9ATXYOJB2Cn38Gi4X3V66jpLIGAE0DJ4PGX3oE1Xu6s5MBXyfnVl+2aJ+kTEIIIYQQrSsjg2IXMwDlTs4wahRceCFWq05Vjc6NwyKZNaE7ug4juvjj5Sab5ETLkWBYCCGEEK0rI4MSkxqUUeTiDpddBkBuaSWVNVZiAty5ZUQUQ6J8uXFY5IleSYgzJmUSQgghhGg9NTWQlUWJQWV7i1zMcPnlgBq7DBDs5Yabs5H5dw5vs2WKjkMyw0IIIYRoPTk5YLVSVJuPK3r+JejSBagbDMt4ZdF6JBgWQgghROtZvpwiZzf7w2Kzp/1+WqEKhkMkGBatSIJhIYQQQrS8oiKYORNmzCB90EjH4fJq+/2MgnKMBg1/D5e2WKHooCQYFkIIIUTLGzMGPv8cnniCtI/n2A8XllfZ76cVlBPo6YLRoLXFCkUHJcGwEEIIIVpWYSFs365GLr/6KpnlVgDcTEaKazPDpZXVbE7IJcLH3JYrFR2QBMNCCCGEaFmpqeq2dqNcdnEFANEB7vYyiVd+2s+x3FIevLBrmyxRdFwSDAshhBCiZdmC4dBQALKKKjA7GwmyuFJUUcXKA5l8teEYM0dFMaKLfxsuVHREEgwLIYQQomWlpKjbOsFwgKcLHi5OJOaU8ti3u4gN9uTRi7u34SJFRyXBsBBCiHPb2rUwaRKUlbX1SkRTGskMB3i44OnqRFF5Nfmllcy+tj+uJmMbLlJ0VBIMCyGEOLe99BL89BN8801br0Q05t134f33wWIBDw8AsoprM8OuavDGBd0D6BFiactVig5MgmEhhBDnjsJCNc63Lh8fdXvLLfDKK62/JtG0PXvggQfg2DF7VhgcZRJHMosBuKhnUFutUAgJhoUQQpxFqqpg0yawWhueKy+HmBiYPbv+8cREdRsQAE8/DT/84Di3fTvk5bXcekXTdB0efVTdAmRkAFBRXUNBWRX+Hi7MGN4ZX3dnJvQKacOFio5OgmEhhBBnjy+/hKFDYcSIhgHxxo2QnQ2//FL/eEIC3Hqr2qTVpw/ce6/KIFdUwMiRKkAWre+jj2D5ctVbGMBkAiCnuBKAAE8XRncLYNvfLsLLbGqrVQohwbAQQoizyJ496nbjxoYZ3ZUr1e2mTSrbWFYGv/4KaWnQubMKtj75RG3Weuop2LlTXfPjj47spGgdhw7BI4/ARRfBs8/C/Pnwyy9kFVXw3KK9AHTyleEa4uzg1NYLEEIIIewOH3bcz8sDPz/H499/dxx/8UUV+NpadnXurG6HDoX774d33oGCAnXs2DHYvx969mzp1QtQpS433AAuLmr8ssEAV1/Nsj3pPDl7FSWVNTwxMZYRMX4nfy0hWoFkhoUQQpw94uLsH6fXywxnZ8Mff6gWagDPPw8REY7zkZGO+y+9BOHh8NVX4O6ujh1fWiFOz2OPwZNPwhdfwJIljWfcX34ZNm9WZRJhYQDUWHX+Om87IV5u/PTAKO4aE4Omaa27diGaIJlhIYQQbWP/fggOdnSDqKmB+HgYMkQFvnWD4XnzoLpaBVpDhqja4Msvh//8B2bOrJ/19fSEDz6ASy+FMWPUax061Lrvrb365z/rP77wQvjXv9T9m26C9HTIzIQbb4Srr7ZfllVUQUW1leuHdqJLoGcrLliIk5PMsBBCiJb1888wcaKjzAHgwAHo318FtraBDElJUFkJgwerx8cHw/36qT/PPgtTp4KmwW23qSDa/7gRvpMmqSDtsccgKgqOHm3Rt9ghFBU57pvN8I9/wNat6ud44YWqZGXMGLjgAlWmUkd6YTkAwRbXVlywEKdGgmEhhBDNr7zccf+jj2DZMhg7Fq64QgXCt98Obm4qAH7tNXVdXJy6HTJE3dYNhg8dgmHDGv9ahib+KXvoIRWYRUerjLM4MwcPqltXV7VBcdYsVeN9771QWgpz5sDcuaokxcur3lPTC2qDYS8JhsXZR4JhIYQQzWv7dhUMrVmj2qOtXAnXXqtqeVesgB491Ajld9+F8eNh8WJVe2rrFzxggLrNz1e3VVWqZjg4uNEvl1dSybI96U2vJypKtV+TjhInVlQE110Hu3c3PPf992pTHMC2bY52dX5+8PbbqpXdhAlNvnR6gRqVLcGwOBtJMCyEEKJ5zZ6tyh2WL1eBVW6uKlt4+mmV4b37blV7esMNMGWKClT37IHkZPX86GjVicCWGc7KUoFsE8Hw68sPcNdXW9ma2MRwjeholalOP0HALFQ99rx5anNcXVarapNmywzHxDR87gk2wx3NLmFXSgEmo4av2bn51itEM5ENdEIIIZpPeroKqADWr3e0Rhs7Vt2GhsL77zuut3WHWLRIBcNBQeDsrDbV2YJhWxDbRDBcVF4NwLu/HebpST3pEuhR/4KoKHUbHw8hMumsUXFxjsl+v/1W/9zKlfVrrp1PPaAtKKviivf/IK+0ijBvNwwG6SAhzj6SGRZCCNF8Pv5YlTVcdJEajvHzz9C1q2p11piQEFUjbAuGbdfVDYZrx/haAwO568ut/Lo/o95L5JaoiWYrD2ZxydtriMsstp+rserotmD4yJHme5/tzSOPqCD3nntgxw7IyXGc+/RT9fNYtkz9+RPeXxlHXmlVMy9WiOYlwbAQQojmUVmpWppNnAgzZkBxMSxdCuPGnfh5U6aowHnHDlZ3G0rs35byRv/LG2SGDzj7sGxvOt9tS6n39KS8Ui7uFcT8O4djdjYy69ud1Fh1rFad0a+v5KNUgyq72LWrJd71uW/FCvXLyNNPw/Tp6tiqVeo2Jwe++061Srv4YvXnFG06mssna+I5v6vq9JGSX9bcKxeiWUgwLIQQonksWKAC1wcegMmTHcdtJRJNmTJF3aanszGoG+VVVt7rNJLColL7cYB1Beoj9m3HHLXB+aWVpOaX0yXQgyFRvjw3uSfbjuXz+R9HOZJVTEp+GQt3pau+xNu3N9tbbVeeflrVAT/0kGpr5+4OP/0EP/wAX36pfsm57bY/9ZIFpVU8OG87nXzNfHDDQG4e0ZkPpp/XQm9AiDMjNcNCCCHOXHGx2hTXrZvqEGEwwOrVqgXX+PEnfm7v3mqcckICR939QZUAU1BSiQVUMGyxsC5J9blNKyjnp91pfLk+kfXx6uP8CB8zAJf3D2PJzjT+ueIg+bUfzx9ILyJ14HBC53+lNuLJ5DOHmhrYuRMeflhlzwHOPx8++0z9MZtVGUvfvqf8krqu88R3u8gsquC7e0bg4eLE81N6tdAbEOLMSWZYCCHEmZs8WWVen3/e0ff3/PNVezXbhLmmaJo9OxxvdGx+KygoUa+3YwcloRGsP5JDvwhvAO75ehuHMx1DICJ8zbUvpfHy1D6YjAbeXRmHU+2Grd+jB6myi6Sk5nm/7UVKiqrxjo52HBs+3HG/tFRN+Kv1+R9H+XpjIuvistmfVtjoS87dlMTSPenMmtCdvuHeLbVyIZqNZIaFEEKcmcpKlQWeNUv1qT1FGYXleLmZcDUZ4eqrsb79DgnVTvQN92JXcgEFfQbAiy+CrvPjVfdSVlXDUxNjmbc5ia5BHtw8ojPvrzzCuyvjiAlwBNHBXq787dKezPp2F0OifNmRlM9h7zB1cvt26NSpub8D5y7bMJK6wfD116uOIHffDf/7H0ybZj/1wuJ9ABgNGlZdZ+qAMB4d350aq87zi/aSXVzBwYwizu/qz8xR0QhxLpBgWAghxJmJj1e9aHv3bvT05oRcvlyfyO3nR9MnXE0mS8otZcK/VxPp586Xtw3Bb9Qo0g8lUP7ZbvpHeLMruYD8196EOW/B8uV8nxlEtG5iSJQvQ6P97K/9yPhu3DSiMwGeLvW+5tUDwzmUXsSgzr7klhzimNFZZay3b4fLLmu578W5xtYyzdZxA6BLF9ingl7uv99+uLSy2n7fqGncOCyS/9t0jB93pRHg6UJBaRWxIZ5E+Jh585p+0kZNnDOkTEIIIcSZOXRI3XbrZj9UUV1jv//FugQW7Uxl8rtr+eu87SRkl/DEd7vQgcOZRbz/u2p5dlRTpQ79a0shCsqqICAAbriBw6UwNMoX7bh6X03TGgTCtuPPXNqTCb2D6eRr5lhBhVqfbKKrLz5e/ZJwCtny+KwS+/1bR0Xx/JRe/PbIGCb2DiatoJx/XNWX/901gp8fHkOgp0yaE+cOCYaFEEKcGVsw3LUrAHtSCujz3Ao2J+Riteqsi8tmYu9g7h0bw7I96Vzwz9/5Iy6H56f04vyuASzfm46u6yTnqe4RfWuzx/llqn9wdY2VnJJKAk4zwOrka+ZYbinWAQMkGD5efDxERIDJRFF5Fff937ZGa4GX7Unj/rnqe/fzQ6N5YmIsAOE+Zv49bQD7XryYS/rIQBNxbpIyCSGEEGfm8GHw97dvlPt4dTyVNVZ+P5iJm8lIXmkV43sFMXVAODcMi+Td3+Lo7OfONYMisFp1fjuQyb60QlLzy9E06OTrjrOTQWWGgZySSnQdAhvJAJ+KSD8zFdVWMvsOJnjuXNU718/v5E/sCOLi7CUSb/1ymCW70gj0dOXZyT3tl9RYdf6+ZL+9T3Ckn3uDl3FxMrbOeoVoASfNDGua9h9N0zI1TdvTxHlN07S3NU2L0zRtl6Zp0khQCCE6kkOH7CUSqfll/Lg7DYAtCXmsjcsGYGSMGrwQ4uXGy1P7cPtotbnqwp5BGDRYsTeDtIIy/D1ccHYy4O1moqC2NVpmYQVw+sFwp9rg7VhMbXuvE2WHrVbVF3nRotP6WueU3FzYuhWGD+dgehGfr0sA4I/an5nN6kNZ9QZmODvJh8qifTmVv9FfABNOcH4i0LX2zx3AB2e+LCGEEOcEXYfdu6FHDwDmrE9A13XG9wxiZ3I+vx/MpFuQB4GWxksc/D1cGBTpy/K96aQVlBPqpa7zcjPZM8OZReUAjdYGn4rI2rZrCQER6sCJguGCAvj9d1i58rS+1jll6VKoqUGfMoW//bAHT1cn7hwTzcGMIvv3HODrjYkEeLqw4O4RfHvX8BO8oBDnppMGw7qurwZyT3DJZcB/dWUD4K1pmhQOCSFER3DsmCo7GDiQkopq/m/jMSb2CeHyAWGUV1nZEJ/LqC4BJ3yJ8b2COJBexM6kfEK83ADwNpvsQzOyimozw00E1CcT4WvG7GxkX6FV1ceeKBjOrf3nrnbqXbv2ww8QHMwPzmFsOprL4xNimdw3FIB1cWqYSWp+Gb8dyOSaQeEMjPRhUGfftlyxEC2iOT7rCAPqdjFPrj0mhBCiPdmwQU0jswWM8fHw3/+q+wMH8r8tSRSVVzNzVBTjYgPtH6eP7HLi+tyLewUDUFheTYh3Y5lhFQz7ezif1rKNBo1eoRZ2pxTAiTbRWa0qsAdISzutr3XOqKiAZctg8mTe/u0IfcO9uHZQBD1DLHibTSzckcK/fj7EnHUJ6MC0wdKbWbRfrbqBTtO0O1ClFHSSpudCCHFu+eIL2LwZFi5ULc+uv16NYQYOB0fz0dc7GRjpw4BOaiPdigdH8/kfRxnZxf+ELxvha6ZHiIX9aYWE2MsknNmfpibMZRaV4202ndEmrd5hXszblERN/wEYFy+GkhJwP24j2Jgx9vfT7oPhVaugqIiKS6dwdH0JD4zrau8LPDLGnx93p/H7wSwAxnYPsE/4E6I9ao7McAoQUedxeO2xBnRd/1jX9UG6rg8KCDjxx2ZCCCHOIrquMokAzz6rBlfUtlIDmPnNbiqrrTwzqYf9WGd/d164rLeaMHcSF/cKArCXSfh5OJOSX8Y1H63nj7ic0948Z9M71Iuyqhriu/RR7+Xw4Ybvb8sW2LFDPW7vZRKLF4PZTPKAYei66rhhM6hz/fHZ1w+NbO3VCdGqmiMzvAi4T9O0ecBQoEDX9Xb+K7UQQnQwBw5AYiKEhEBKClxzDXz+OWzbRpHVQOJPeTw+IdaeFf6zrhgQzrI96faBGzNHRWEyavx+MIuj2SX2YPl02Sbf7Xb2pStARkb9C3JyoNyxaYzCQigtBXM7zYiuWQMjR3Ks1AqoXsw2VwwI52h2CVcNDOeX/ZmM7S7JK9G+nTQY1jRtLnAB4K9pWjLwHGAC0HX9Q+An4BIgDigFbmmpxQohhGgjtqzw8uVqhO/kyaBpMGoU8Un5wB9EBzTsP3uqOvmZWfbgaPvjQIsrj10cy2MXx5JVVIHZ+cz62MYEeOBqMrC7yoUroGHmNymp4ZPS0yE6+oy+7lmppER1AHnqKY7lqEEnnepkhr3MJl68TI3W7hvu3SZLFKI1nTQY1nX9upOc14F7m21FQgghzj5Ll0JsLPTpo/7UEZ+t6mxjziAYPpHTbalWl9Gg0TPEwt5ClQltEAwnJzd8Ulpa+wmG09JUVh9Ub2GrFYYOJTGnFDeTkQCPM/8eC3Guks7ZQgghTqykRG24mjix0dPxWSUYDRqdfFsmGG4ufcK82JtejNXTooJDXXecbCwz3F420a1ZA2FhjnroDRvU7dChHMstpZOvGU3T2m59QrQxCYaFEEKc2MKFUFkJkyY1ejo+q4QIH7ezfjJZ7zAvSiprN9G99RYYDCpDCvUyw1sGjaPAxV21jmsPtmxRgf/atao2evZsGDgQ3d+f/WmFZ1TeIkR7cHb/n0sIIUTb0nUVPHXvrsYUNyIus5joAI9WXtif1ztMbaLb27mX42BJibqtzQyXG01cN/av/Hf8zSqj2h4cOKBut2yBO+9UAfFnn7E3tZCU/DLGdg9s2/UJ0cYkGBZCCNG43bth9GhVY/rooyqTepyyyhrisorpHWppgwX+OV0DPXBxMrA7MMZx0NZXODkZunWjwM2TKoORlOgeKhiuqWmbxTYnWzD8zTewYAG89BL068fyvekYNPhLDwmGRccmwbAQQoj6Xn0VLrlETWvbvx8+/RRuu43iimoWbk9Br1Nruy+tgBqrTp9zoOuAk9FAjxALu13qTMQrUoM9SEqC/v3J81Ct4TL8w6CgQP1CcK6zBcPl5XD++fDIIwAs35vO4M6++MnmOdHBteoEOiGEEGe5nBx4+mlVHnHXXSqL6KeCxznrEnhj+UGCLK70DLXwxIJdLN2jujL0re3je7brE+bFwqPeWNEwoKtgWNchOZkvr7iPtHsuhRLIdKkt+9i2Dfr3b9tFn4m8PMjMVH2hDx2COXPAaCQ+q5hDGcU8N7lnW69QiDYnwbAQQgiHhQtVcLh1K5x3Xr1TKw9kAvDpmniOZpeQkFNiPxdkcW3VZZ6u3mEWvnRyJck7iMj8dBUMZ2dDRQVvGqMpKnMCdDIqdNVHOTGxrZes6Dq88QZcdx2UlUG3bqf2PNukvenTYcoU++Hle9XQkfG9gpt7pUKcc6RMQgghhMP8+RATo0ok6sgrqWTbsTyMBo1fD2RSWF7FvDuGM3NUFA9e2LWJFzv7hPuo4RJp/7dAHSguhuRkSk0u5OtGaqyqBCSnpJLq8IizJxhOTobHH4dOndRmxsWLT+15KSnqtlOneofXHM6iR4iFMG+3Zl6oEOceCYaFEEIoWVnw66/qI/Xj+s6uOpSFVYfnp/Ri6oAwFt8/iiFRvjxzaU8evPAUs5RnAdsAj0zdpA4UFUFSEqme9UcO6zpkd+0JCQnqwLffQkQEVFW14mrryMur/3jVqlN7ni0YDg21H6qusbIjKZ9Bkac3OluI9kaCYSGEEMr//qe6J1xzTYNTvx3IxM/dmelDOjH72v6EeJ2bGcVAWzBsra0SLCqC5GTSLP4Nrs2M7OoIhu+/X2Vnj59c11pyc+s//uOPU3teSgqYTODveH8HM4oorazhvMizf9OjEK1BaoaFEELAk0/Ca69Bz57Qr1+9U9U1VlYdyuLCHkEYDOf2pDIvNxPOTgayqmtzQcXFkJVFqk/D2tmM4EgVAFdXg3vtYIqMDJUhbm22YPjxx1XN8Pvvqx7J7scNzPjvf+HHH9X1xcWQnU1c9wHc9uYq/u/2YYR5u7HtWD4AAzv5tvKbEOLsJJlhIYTo6DZuhH/8A6ZNgxUrGpRIrDuSQ0FZFeNiz/1+tJqmEeDhwpytqcy4+gX0wtoyidBo+zW+7s4ApPiGqEx5Sgp41HaXaKvMsK1M4p571Ea46mqVyT/e00/DL79AaSls3w5xcazvNpjEnFL+OJxNeVUNX29IJNTLlQjfczO7L0Rzk2BYCCE6sqoquOMOVVP68ccQFtbgkk/WxBPg6cKFPc/9YBgg0OJCeZWV1dEDKSsugeRkUgPC7ee7BXkQbHFls7G2pjYhoe2DYVtm2McHxo1T2ftXX60/FKSqClJT4d57VRnF0KEAxAVEArAzOZ9/LDvAgfQiXp7aB007t7P8QjQXCYaFEKIjKSxUQzSyslTw9NZbsGsXvPMOeHrWuzQ1v4z7525nzeFsbh0ZhYuTsY0W3bxsdcMAxSUVKjPsFWDvrOBjdmZ4jB8bCzR0UMGwrRwhLa3V1wuozLCTkwrKNU1lgA8dUhv7bFJSwGp1dI4YNAiAOA+1OfDH3Wl8/kcCN4/ozNh2kOUXorlIzbAQQnQU6ekqQEpJgfvuUx+1G43qY/fLL7df9uWGRLYl5rFsTzpWXeeBcV2YeX5UGy68eTkZHXmgkpJytYHOxYs+YV6kFZThbXamf4QX329PIc4vgq4JCVBZqZ7QlplhX19HCcsVV0BsLLz8Mlx9tRqVfeyYOmcLhnuqgRqHUYF8fmkVscGePDExtrVXL8RZTYJhIYToKB57TGWEv/pKteYqK4P161VWuDbIKiit4vlFe6mx6kzqE8ITE2OJ8DW38cKbV2p+mf1+SVomemUlKZorf/F1475xXRkW7UugpxoisqPHELomJjrGNqelwfffq6lud96xJDt+AAAgAElEQVTZeovOy1MlEjZGIzz1FMyYAUuWqF9obD2RI1VZBKNGUehsJtPJzF9iAzmQXsTb1w3A1dQ+MvxCNBcJhoUQoiNYtUoFwU8/raaRTZ/e6GXL9qZRY9X54d6R9Iton623npgQy7UfbwCgODmNPDcLFRgI9XbjlpEqA15RXYOmQXJ4F0jY5AiG09NVmcnhw60aDOu5uXzYdxKX5ZdRUlFNl0APtOuug+eeg1deUcGwLTNs63bRvTtHt+2HL3dz7eAImTYnRBOkZlgIIdq7qiq1qSoyUmUTT2DRzlQi/cz0DfdqpcW1vqHRfiy6byQAxVU6qRZVU1u3d7KLk5FgiytJ/mGqZri4WJ1IT4fycjXCuRWlldbwj+hxjHjtNy6avZqf92WoGuIZM1Q3kJdegmeeAS8vMDsy+Wm1w0XCfKRzhBBNkWBYCCHau7ffhr171a256ZKHzMJy1h3J4bJ+oe2+04C7i/pgtMTZjZTaYPj40cQRPmaS3f0gKYn5oQPo/PgSKrOyVTCcl6dqrltJfmn9yXcb4mu7S9hKIv72N3VbUFDvurSCcgCCLa4tuj4hzmUSDAshRHtmtaoewhMmqI/Sm1Bj1fl64zF0Hab0D23yuvbCozYYLnZ2I81TTWcL8a4fMIb7upFkdIfqav45RE3lyzS4qlprgJycVltvfqW13uPNCbXBcLijJRyaBgsW1LsuvbAcZ6PB3jtZCNGQ1AwLIUR7tmeP2jQ3bdoJL3vomx0s2pnKoEgfugR6nvDa9sCeGXZxIzsiBmcnA37HBYwRPma+r3GiwuiEe5XKsGa5+xBuy75mZ0NQUMsvNi6O/BqVqX9uck9yiit5//c4Ckqr8Ko7De/ZZ3nHqw9L31pDQVkVZVU1VNVYCfJyafeZfiHOhGSGhRCiPfvtN3U7dmyTlyzbk8ainancMTqaObcOaaWFtS1zbUeF4vseJHXCZYR5uzUIGCN8zehAqiUQc6XKBmd6+DqmwbVG3XBVFVx/Pfm+qi/wxN4hjO8VhFWHb7Ycqz8kpXNn5qxPpLiimqFRvlRWWykqrybEIvXCQpyIBMNCCNFelZWpoQwxMY7es8cpKK3ibz/spWeIhccu7m7PmLZ3BoOGu7OREh9/Uis1Qrwa1tRG1G46S/IKsmeGMz18IT9fXZCV1fILffZZ2LyZ/OtmAOBtNtE33Jvzu/rz0ap4Sl3c1KY5oDy8E9nFFVw9MJx/XdufgZGqFVtQI+9NCOEgwbAQQrQHa9eqLOGOHY5j06ersbx//av90JaEXP46bzvL9qSz+lAWf/9xH7kllbx+VV9Mxo71T4K7ixMlFdWkFZQT6t0we2rrr5zUqRsu1WroRqaHL+i6uqClM8O//abqvWfOpCCmOy5OBnuP4Acv7EZOSSVfrk+0t1JL8VO13rbOEbbWeJ6uHeMXHCFOl/wXIoQQ7cHPP0NqKlxzDWzZorKWCxeqdlv33w/AtmN5XPPRegB+2JFqf+rdF8TQO6z9tlJrioerE/mlVWQUlhPaSPY0yOKKyaiRFBZNuaZGOGe61xl80ZLBcE4O3HgjdOsG//43eUvj8Dab7KcHRvowulsAH62O54ZOUbjv20eyi/oZhvuoID7KX93mFFe03DqFaAckGBZCiHNBSQkUFkJISOPnd+5UE8ri4+GOO9R1BgPcfTcAVTVWnvpuN4Geriy6byRbEvOorLayIymfv/6layu+kbOHh4sT8dnFWHUazQwbDRqh3m4k+YVRXKy6OWR6+PLqBbeQ7BXIey1ZJvHAA+oXmiVLwN2d/NIqfMz1N/g9eGFXrnh/HV/2Gc9dyYmkFKn2a7bM8F96BDGksy8PXtit5dYpRDsgwbAQQrS1oiKYPx/GjIEuXRq/5tZb1TXnnacGZ1x5Zf3zO3bA+PHQvz88+aQ6dvPNEKo+Ov9s7VEOpBfx0Y0DCbS4ckkfFVRfPiCMjsrd2YldyaozRGPBMNT2Gs7xo6RSDd3I8PClRjNwxC8Csn9pmYXt2wdz58Ljj8OAAQDkl1Xh5Waqd9l5nVR2+NNUZ675dRWbf03AyaAR5Kmy2BZXE/PvGt4yaxSiHelYBWJCCHE2yc1VG6QiI2HmTBgyBHbvrn/NihWq5nfpUhg8GCorVSnEwoWOa/LzITFRBcKzZsGkSeDvr+pNgaTcUv79yyEu6hnExTKS167uZsFQ78Y3mUXU9houcVbBcqa7LxVOzuSYLS1XJvHOO+DmBo88Yj9UUFpVr0zCZnRXf7KLK7lt/l6+355CtVXHqYPVfgtxpuS/GCGEaCtTp8Lf/w4XXAA//KAC3ffec5zPzVWb4N5+W2WPZ82C9etVUDxtGvz+u7rOtmmuXz9VGvHDD3D4MASqdlwvLN6HUdN4YUqvVn17ZzsPF6P9ft1RzHWF+5jJrTGQ76E2oxW4eVDhZKLc5Eppbn7LLGz3bvUz9ve3H8ovq8TbreHgDNu6tx9TaxkR49cyaxKiHZNgWAghWtubb6qShtWr4cUX4bvv1HS4CRNg8WI1NQ7Ux+R5eSoo0jQYNw48PODHHyE6Wj1n717VSULTYOhQ9TyjEbxV8FZYXsXKg5ncOLxzk6UAHZVHbZcFs7OxyZZyto4SNTqYDFBlNFFqUt/HFtuYdvhwvXIZq1Unr7QKb/eGmeG6U/OuOC+MD24Y2DJrEqIdk5phIYRoTVVVqnzBtvnqiisc56ZMUeN0N2xQAfGnn6qPykeMgG3bwNdXXefnp8onYmJgzhx1rk8fx/k6NhzJocaqc0H3gFZ4c+eWqQPCSc4ro1eopclrbL2GAfzdnEgrqSbf1QOA3LIaIpp64uk4elT9YpSZCV0dmxo/WRNPZbWVXqENO36E1fkFp1eoV4O6YiHEyUkwLIQQrWXhQnjwQRUIu7tDcDD07Ok4P2UKBASozXIGg+of+/zzKhtcN2gGCA+Hvn1V2cTWrXD77Y1+yT/isnEzGRnQybvl3tc5amCkD1/ccuKJe7bMMIC/uzNpJdUU2IJho6vq8uHu3jwLuukmWLNG3a/NDG9NzOP15Qe5pE8wk/s27CTi7+GCk0Gj2qoT7iOZfyFOh5RJCCFESysrg7vuUjXCOTkQG6vqfH/8UZU32Hh7q4lxqamwf7+qH/bwaPp1Bw1SJRJlZWSPvIC3fjlMeVWN/fSCrcnM3ZTEqK7+uDgZm34d0SQ/d2fcagdd+NV2aagwqdtss1fzbqKrqFN20aUL+aWVPDB3O6Herrx2Zd8G46JBtX8LsqhSiQgfc4PzQoiTk2BYCCFa0p49ajPURx+pDXA5Oap1Vpcu0L17w+tHj1Yfkx85ApMnn/i1BzrqQxcF9mb2L4d4ZuEedF2nrLKGF5fso2+4F/+4sm8zv6mOQ9M0InxVxtXPUj/zmtvcwXBlpf2uHhPDo//bRWZROe9dfx4W16bLH2ylEuG+khkW4nRImYQQQrSUxES1qc3DA5YvV5vmToWrq9ogdzK2YPjSS9mbWQrAt1uTOa+TmpJWUFbFrAmx+Lo37EIgTl2Ej5lDGcX4e9Zvv3bKwfCqVZCWpjqANEXX1cAUT0+YMIEvdmbxy/4Mnr20J33DT1ziEubjhnem6YQBsxCiaRIMCyFES5k/H0pLVUlE1xNPeUvNLyPA0wVTEz1iP/j9CMv2pDGmeyDXDApXI3f794evvoLJk9n7nx2c39UfTdN4ftFeLG5O9A33YnBnn0ZfT5w6W92wv4dLveM5ZotjI+SJvPkm7Np14mA4J0dNGJw9m4I77uVfr//GmG4B3DKy80lf/v5xXbjivI47PEWIMyXBsBBCNCddh3XrVMeHBQtU9raJQFjXddYfyeHjNfH8fjCLAZ28+eymwfUyucl5pexNLeS/6xMorqhmd8ph/m/jMebfOYzoAA+YPp2K6hriMosZFxvI7edHc+k7a0ktKOPTmwY3Wmcq/hx7MOxZP8Oe63aKmeGcHMjIUH83mvp5xMer2+hovliXQFF5NU9MjD2ln190gIf6uyCEOC0SDAshRHPZvx8eeAB+qTOm9+WXm7z8iQW7+WZLEv4eztw0PJKvNh7j0zXxzJoQC0BltZWZc7ZwIL0IgFem9mFwZx+mfbyB6Z9uZP6dw4nwNXMovZhqq06vUC983J2Zd8cwEnJK6B8hHSSaw9QBYWhAzHEBZ46796kHw+XlUFysyiCqq1Uv6LqB7pEj6jYmhi0bcukT5kWPkKZbvgkhmo9soBNCiObw8ceq1dmWLTB7Nrz7rto098ADDS5dtiedV5fu57vtyVwxIIy1j4/jhct6MzTKl+V70x0vufoIB9KLMBlV0DS6mz9dgzz5721DKKmo5obPNpJZVM6mhFwA+te2T4vwNXN+V+kr3Fx83Z25dVQUrqb6HTlyPXxOrUwiV/18yMhQQXFMjOoEkpTkuMaWGY6KIj6rhJiAZmrXJoQ4KckMCyFEc/jwQ+jdWw3DCGg6ELVadV7+aR9JuWUATB/WyR5kXdwrmOcW7eVwRhGapvH2r3FM6htC3zAvNifkqjph1HCFObcO4coP1vH1hmPsTS0k0s9cbwCDaH5udYJhP7OJ3GoLHM2G/Hy1SdKpkX9Sdd0RDGdmqoEqx46pP3PmwDPPqHNHjpDbuSvbE4tILSgjyr9Zx3kIIU5AMsNCCHGmdF19zD1y5AkD4a2JeUz7eANJuWUYDRoBni4MiHBscJvQOxhXk4FZC3bx+IJduDkbeX5yL+4cE8OnNw2u91oDOvnQLciTbcfy2Hg0hxExfi329oTiYnL8kxnq5UqJkwvlWTng4wMzZzb+pIICqKnt/ZyRoT4t6NJFDerIyXFcFx/P4xfezW1ztqDrECWZYSFajQTDQghxpnJzVSeAmJhGT1fVWHlzxUGu/nAdmxJyCfR0Yd4dw3j3ugEYDI660SCLK7Ov6c+u5AK2Jubx9KQeBHi6NPqaAH3DvVhzOJui8mqGRUsw3NLqlkmE1Gbpc5Nry1rmzGn8SXUD3pUr1ZCUO+4Af39HxhjgyBFSLY5fpKL9JRgWorVImYQQQpypOp0AGpzKKubBb3awK7mAqweG89QlPXB2MuDu0vj/fif2CWF9pA/HcksZGHnitmj9IryZvyUZgIt6Bp3ZexAn5Vpnil9obUlKbmk1oSd6Ut2A95NPwGSCm2+GuXMd58rLISUFg7Mz6OpQZwmGhWg1EgwLIcSZqtMJoK7kvFImv7MWk5OBD6afx8Q+Iaf0coEWVwItrie9rl/tMIbR3QIwO8v/zluayahh0K1YNQOh3urnk2P2clxgtYLhuA9cc3LYGN6LTA9fJh9YA9deq0ppfH0dwXBCArquk6C5oQEXdAvAo4lfloQQzU/+axNCiNNVVgZubvU6AdS1aGcqJZU1/Hr/qAZtuZpDr1AL/7y6n2SFW4mmabjpNZRoBkK8ajPDbnXanyUlQWRk/Sfl5PDJkCvYHxSlguE771THfX1h9251Pz6ePDcLRVYDf7u0J7eNqv/3SAjRsqRmWAgh/qzSUrjySggNVRukjhyBoCBwd6fGqrN8bzob43NYsjONfhHeLRIIgwrOrhoYjpebjOFtLa7OqlTCViZRLzN88GD9iwsL4bffyDVbyHL3Qe/eHS64QJ2rmxk+coQEH/WpQWc/c0suXwjRCMkMCyHEn5GRAVOmwKZN6vGePWrYRmws6+KyeXHJPvuQDIBnJvVoo4WKluDq4Q75ZQR6uuCkQY65zmCTDRtg/HjH4+eeg//8h/yZH1JpNFG0ZBkWTSO7uAIP3wBcc3NVJ5L4eBKDOgMQ6Se1wkK0NgmGhRDiVOXmwvDhkJ6uBms89BDs3Qt79vDv217k359uJMzbjbevG4BR0zAZNcZ0l+EX7YmtvZqLyYCvi4FcswWCg6F/f3j7bXj4YdVzGFRmGNQ1QJaXP5RXMe6fv4M+mP95h9H9q6/gwAESO/dA0yDCV3pFC9HaJBgWQohT9b//wdGjqkXW6NHw9NOwbBkUFbHUI5LBYT58edvQBpPKRPthG7zhajLi6+5MjtmL3V0H0GnWk3iNGw2LF8N116mLDQZqNAMFtXXF2UUVrDmURWF5NWDkp+4j6T5jBgCJd11MqJcbLk7yd0eI1iY1w0IIcaq+/14NTBgzRnUN6NEDvv8eKxqJNc70C/eWQLids/18XZwM+FncyHL3YfKIe5mxs3awRt3xzKWlFMT2tnVLI7Oogi83JNIvwhuLUSffzdN+aYJnIJFSLyxEm5BgWAghTsWcOfDbbzB1Kmi1gzJ69QIg08OXcitESm/Yds/VZEDTwNlowNfiRpx/JwB2ppeoCwoKHBeXlJDn7W9/uHhnKkeySpgxLBIfFwP5ro5gONHJQ+qFhWgjEgwLIcTJHD6sBiX06QP33+84PnEimM0kXDABgEhfyey1d65ORlycDGiahp+7MyXOdWp8zWbIz3c8Likh3+Jrf7hiXwY+ZhOT+obg7WoirzYzXOhsJtdqlMywEG1EaoaFEOJklixRtwsWQESE4/i0aTBtGsc2J8GCXXSWzF6752oy2kslfN2d650r9/XH9bjMcG5I/Z7B1wyOwNVkxNvbnTw3C3z9NceyyyFV2qoJ0VYkMyyEEE2probrr1cdAmJjoXNn+6mk3FIW70wFICGnBCeDZp9KJtovdxcj5iaC4YTQmPqZ4dJS8txVH2KLqxOaBjcMVUM5fCxu5PXqC9df7/hkQX6ZEqJNSGZYCCGa8uqrMHeuuj9tmv3w+iM53P31VvJLq+gf4c3R7BIifM04GSW/0N7dfUEXpg4IB8DvuGA4LiiK2IJj9sdbXQKYFXoBAFP6h1Jj1YmoLaXxNjuTX1oFQGJOKYCUSQjRRiQYFkKIxmzeDC+8oDLDb78NXirDN2/TMZ5ZuMc+9e1gehEH04voHuR5olcT7USUvztRtRslbZlho0HDbDKyOKwfl27ZZb/2884j7Pf/fllvNNvGS8DbbKKovJrqGisJ2SUEeLpgdpZ/koVoC5LGEEKI45WWwo03QkgIvPsu+PmBkxPbj+XxxHe7GdnFn0X3jwJgV3I+R3NKiA2RYLij8fNQwXC4jxszz49muVcMe3RHqYNPiSqZ6B1mqRcIA/iY1XPHvbmKX/ZnSL2wEG1Ifg0VQnRMlZWqFthohMhIVRM8YYJqm/bgg3DwIPzyC/j42J+yeGcazk4G3r1+AJ6uJkK9XFmyKw1dh9hgCYY7Gl93FwD8PVy4alA4s385xG5Xf3rXni/FiVAqWHTvqAbP9TarTxaO5dpKJKReWIi2IsGwEKJj2rQJ3nsPXFygokIdGzoUxo+HTz6BJ56Av/zFfnlxRTVL96QxumsAnq4qkOkW7MnvB9WQhdhgS6u/BdG2vN1MGDQI8HDBu7ZsptBqAF2HqirKnJwxG3QMBq3Bc22ZYRvJDAvRdqRMQgjRMR08qG737YPcXDVUY9cu+Pvf4eqr4eWX7ZeWVFRz4ZurSCso56qB4fbjPUJUAOzubKST9BjucAwGjegAD7oGeWB2NuKEToHJDcrLoaSEUpMr5ib+lT0+GPZwkdyUEG1F/usTQnRMBw6orHBkpCqVmDFDjVpetQoee0yNW661+lAW6YXlvHPdACb0DrYfv+P8aKL83In0Mzea/RPt3w/3jsS5dgiHxWCl0MVdtVerqaHU5IKbU+N/L2xlEr1CLZRX1TC+V3Cj1wkhWp4Ew0KIjunAAejWTQXCNiNGqD91LNuTxks/7sfbbGJi7/oBi4+7M9cMjkB0XO51MroWJyh0rQ2GjUZKnd0IMDWeGo7wNfP1zKEM6uyDi5Ox0WuEEK1DgmEhRMd08CD073/CS1YdyuKur7YBcFn/UOkjLE7I4myk0MUDCgrAxYVSk4t9QEdjRnbxb8XVCSGaIsGwEKLjqaiA+Hi49tomL8kuruCR+TvpFuTBzFHRjOwqgYs4MYubSWWGDx+GqCjKTK64SS2wEGc9SXMIITqetWuhpgYGDWr0tNWq8+j/dlJYXsU7153HNYMjCPN2a+VFinONJdCXQosvPPcc5OZSanLFvbbziBDi7HVKwbCmaRM0TTuoaVqcpmlPNHL+Zk3TsjRN21H7Z2bzL1UIIZrJwoXobm7M8+vF/M1JrD2cTUp+mf30Z2uP8vvBLJ6Z1IPu0j9YnCKL2ZlCvyA4ehReflllhiUYFuKsd9LPbzRNMwLvARcBycBmTdMW6bq+77hLv9F1/b4WWKMQQpy+RYvUFLmqKqiuhjvugB9+YM/k63liycF6l155Xjiju/nz6tL9XNwriBuHRbbRosW5yMvNRKFugKlTqV74A5VjTZjdXNp6WUKIkziVYqYhQJyu6/EAmqbNAy4Djg+GhRDi7PPBB7Bxo9osFx8Pd90FpaUcevRCSIUvbxuCyWhg5cFMPl4dz4JtyQzo5M3sa/s3GKErxIlY3EyUV1mpeP0NKn75HQCz2bVtFyWEOKlTCYbDgKQ6j5OBoY1cd6WmaaOBQ8BDuq4nHX+Bpml3AHcAdOrU6c+vVgghTkVenuoWMWyYur3kEpg7F778UvUTNpuJ69wTp/RkhkX7YTIaGBbtR98wb9bGZfH0pJ6YnWXjk/hzLK7q70xRcDjVDz8K5eDmJWOWhTjbNdcGusVAZ13X+wI/A3Mau0jX9Y91XR+k6/qggICAZvrSQoh2Y9s2x2S407VvH/j6wvDhald/QgLExqpzU6eChwdceSVx+ZV09nfHVKdd2qS+Ibx6RV+ZBiZOi6V2JHNBWRWl99wPgNlDNl4KcbY7lWA4BajbVT689pidrus5uq5X1D78FBjYPMsTQnQYug5Dh6rA9dtv6587ehReew2s1pO/xl//6ni8YIE61r27euzhAZs3w9tvE5dZTNdAj+Z9D6JDs9Rulissq6K0Sv1dlU8YhDj7nUowvBnoqmlalKZpzsA0YFHdCzRNC6nzcAqwv/mWKIToEHJy1AY3UCUNdc2aBU8+Cd98c+LXWL4cfvkFnn9ePf7uO3VrywzX3q/w8CQxp4QuEgyLZmQbsXwkq4SyqhoAzM4yXU6Is91Jg2Fd16uB+4DlqCB3vq7rezVNe1HTtCm1lz2gadpeTdN2Ag8AN7fUgoUQ7dSRI+rWYoH161VGF2D3bpXh1TR49lnHcVD3r7oK3npLBdKPPgoxMSpwDg1VWWCArl3rfan1R3Kw6tA/wrsV3pjoKPqEeREb7Mm/Vhwkp1h9WCrBsBBnv1OqGdZ1/Sdd17vpuh6j6/rLtcee1XV9Ue39J3Vd76Xrej9d18fqun6gJRcthGiH4uLU7fTpkJYGmzbByy/DmDHg5wevvqqu2b7d8Zw9e1Sg/OCDMG0a7N0L//gHODtDt27qmsGDwb3+Jqble9NxdzbKOFzRrJyMBl66vDepBeX86+dDALiZpExCiLOd/FcqhDg72DLD112n2qENG6Ye/+Uv8NFH4OmpMr5Llqja4fnzYcUKdc2gQSooHjECrrhCHTObAZh353N8+MZKqmp0aqw6M0ZEsmJvBmNjA3E1SdZONK9BnX25amA4325NBiQzLMS5QIJhIcTZ4cgRCA9XQfDVV0NkJNx+uyPDCzBkCLzwghp3a+PlpWqFH3sMHnpIlVMAzJ4N48axyOpHcUUxY7oFsCs5n38uP4hVh0v7hrbu+xMdxhMTY1mxN53C8moJhoU4BzRXazUhhDgzcXGq3tdkUlnfN96oHwiDCngnTYLPPoPUVNUz+J13VCu1zz6D3r0B2J1cwBdZJvSHHyYuUwXCb17Tj8cnxGLVwdfdmXGxgW3wJkVH4O/hwnOTexHt74632bmtlyOEOAnJDAsh2l5WFmzaxNyH/oG26RhXDQzHydjI7+pXXqn+2Mxp2NJ85cFMbvlcbZzrG+FNZlGFvWvEmO4BRPi6MaVfKM5OkgsQLefKgeFcOTC8rZchhDgFEgwLIdreV19hra7hSUN3+G43OSWV3Du2i/306kNZvP97HJ/MGIRnbS/XxlRU1/DCor32xz/tSgOwB8Mmo4HfHrkAo4xZFkIIUUtSI0KI1ldQAP/3f/DSSzBwIDz8MKmjL7KfXns4235f13VeXXqADfG5fLI6vsFLZRVVUFWjBhzMWZdAQk4pH984EKNB48fd9YNhUAGxwSDBsBBCCEWCYSE6moQEOHaseV4rLg4uu0xtaqvb//dkZs9WLdT+9jfVBu3110l88z0AYoM92ZGUT1WNFV3X+WRNPPvTCgm2uPLJmqOUVlbbX6a8qobxs1dx/ScbSMkv451f4xjbPYDxvYKJ8ncnraAcZ6OBCB8ZiSuEEKJxEgwL0dFcfTXccEPzvNZrr8GiRWriW2LiqT9v9WrVOSIjQw3YeOwxjlar8ocrzwunrKqGFxbv5YJ//s4rPx3gop5BvHZlH8qqauxZ4+oaK+uOZJNXWsXmhDwm/ns1ZVU1PHNpTwAsrqoKbEr/0Mbrj4UQQggkGBbnkv37obKyrVdxbisuhm3b1ECLqqrTf52KCvjXv1SpQ1iYOpaefmrPraqCDRtUP+BAR0eHhOwSXJwMXNJXTXf/asMxgi2u/Pva/nx0w0BGdvHH09WJL9Yl8NA3Oxjw4s/c+sUWAGZN6E5heTU3jehMTEDtZrlu6rUfuui4jhRCCCFEHbKBTpz9SktVD9p+/eDSS+G778BJ/ur+aVYrrF2rbisq1Jjj8847+fPKylQJRO0QC0C1OHvnHfDxUdnhG29UU+OOl54OLi7qOpvt29VrjhxZ79KEnBI6+7kT5u3G/80cSoi3G1H+jslxBjTGxQbyw45UvNxMnN/Nn592pzMkypd7LujCqC7+9Aix2K+/d2wM04d1wt/D5ZS/RUIIIToeiSjE2S8iAnJz1TCFxYvhgQfgvfccwxVEQ1u2QFCQ+t6BCnxvuqn+KONNm04tGL7xRhU8L16sHn//vQqEH3pIZYdtQfDxwXBVFfTtq9qmPfywGpPs5ASrVqnzo0bZL7VadfanFdEnzAuAETrbLhoAACAASURBVE2MSX720p5MG9yJgZE+ODsZ2JNSQKBFBbt9w73rXetkNEggLIQQ4qSkTEKc3UpKVCAMMHQozJqlRvW+/nrbruts9v33MHgwdO6sAtjXXlMdG5LVeFi6d4eAANi48dReb9cux7UJCXDrrWr88WuvqWOBgWAwNCyT2LRJBcLe3ipovugi9fjnn6FnTwh1TIDbEJ9DSn4ZE3oHn3Apfh4uDI/xs/cI7h3mRaCn66m9DyGEEKIRkhkWbU/XISlJZTGPz/baso2+vvDuuzBggOqE8MQTavhCly4NX68jKy6Gv/4V+vRRmdnLL1dlEVddBe+/rzo3VFSogHbTppO/nu1nU14OeXkqS2y1wjffqNcCMBpVcG37WX3wgfoaeXkqSD5yBJYsgTvuUEF0RgbcfTcA2cUV/Lo/gy/WJWJxdTppMCyEEEI0NwmGRdv75hu47jqVLVy/HiyOuk97gDV3rspuggqE582DrVslGAbVs9doBA8PePFFFbzOm6cC0ltuUW3PZsyo/4vGkCHw009QWFj/+3283FwVCAOsWaNqjl95BaKjAaistvLbgUy6x/QmKi0Nli6Fe+5RI5Wjo1WG2tdXff1evWDqVKio4KcBF/Hos8soq6pB1yHM243np/TC1WRswW+UEEII0ZCUSYi29/XX6nbfPnjrrfrnbB+9h4Q4jnXvroK/PXtaZ31nm+pq+Oc/4eabVdnBmDEwbZr6fsyeDbfdBiNGwNixqqzhppvqBcJPfreL53wGqazv1q0n/lpJSY77c+eq2+HD7Yc+/+Mod321lXGjHiT/8FEV9EZHq6z0wYMqG2wzcKD6enPm8JMxmNLKGh4d350fHxjF2sfHcsV5MrpWCCFE65PMsGg75eUqgFuxQm3Gio+HN95QH6UbDJCSAps3q2tDQtB1HU3TwNUVunaFvXtP/Prnmrvvhi+/VBvf7r1XbTo73v79KgjetEl9j+bMUcf37lWBscXiqOVtRHWNlUU7UnF1MvI8oG3cqILmunTdETzXDYbnzVO3tZvujmQV89/1qrewrmmk5JfhXVamMsh//7vaLHfLLfVfOyAA/cYb2fTKr1zWP7TeyGUhhBCiLUhmWLSu/Hy46y7o0UN9rD94sOodPG0avP22asF1ySUwYYLKcH74IZhMZLt4MPqNlXy6pnYcb+/e7SszvGmTeq/Dh6ss+KOPqpIEUKUil14K//63qpk+ckQFpmvWOJ5fXa1e4/XXwb/xTgwAB9KLKKmsIae0mqQBw9Xwi7p++QXc3By/aNg23fn5qdvgYLBYWLA1mb+8uYqU/DLuGhMDQKaHn6pLjo1V2f45cxrt+JGYU0pmUQVDonxP61slhBBCNCfJDIvWNX68au81aZKahNanjwrwbLW/mzapAC0kRGVJP/6Yu6c+RfbX20jKLeOlH/djcTNxTa9e8O236jXmzj23+w4XFqryguBg1UNZ01RLsptugp07VUuyH39Uf7p2VUFwUJB67tq1qjb4ppugUyd7Jray2orJqKlMOlBUXsWbKw7xxboE+5fdPnoSnT55Rf0yYtsM99VX6vXuuUd93YMHeXP0DCqvuponH5gMQ4ZwIL2QpxfuZmiUL7MmdCfQ05UPVx0h89GnYMZU++vrus61H22guKKa+8d1YWIfVeqy8WgOAEM6SzAshBCi7Z3DEYQ45yQlqbKH119XQxsaExSkAlyA/HwqPvsPS6MGQ0Ie/cK9sLiZePK73fieP5ELh69QAfG0aaqzxLnIalWBbFwc/PoreHqq43PmqFrgW25RQXC/fqr+98MPHYEwOAZXrFunnmswsDu5gCs/XIefuzP/uXkwGYXlPPXdbtIKy+1PMzsb2RbUh8tKS9WmxTFj1IlDh9Tt6tX22uAVd35EcYmZJ7dvp9AngLu/2obF1cQ71w8g0NOViuoaADK69a731g5lFLMpQbXFu/vrbdw2KoonJsay6lAWwRZXugR6NO/3UgghhDgNEgyL5lVdrT7W/+47R1cBG9uwhYsuavA0ez1wXaNGkeXumFz20EXdGNzZl+s/2cC9f+Sy+sefCRrUV20aOxuD4ZIS9f3w8nIcs1pVuYOzM9x+u9oIt3ChOmYLSAHOP1/VDL/5piodWbgQwsObzoDXDteoqrHy+IJdeLuZqKy2Mv3TjeSWVNIl0IMFd4/AyaBRbdV5Y9lBtpdWqI2IP/+svnZlpRrV/MgjcP/9Kiu9axepZaEU5ZdR0qM3Ty3YxbHcUubePsze39fFyYiP2URGbbC9+lAWFdVW4rOKAVgzayyfrT3KZ2uPsjMpn4MZRVzSO6Thz1sIIYRoAxIMi+Yzd67KZFosajOXwaCCqspK1RViyRI1gKFPn3pPW3Uoi5lzNjOmWyDvXj/A0V7L25uMa2cA8PnNg7mgeyAAL1zWm8vf+4PtKUVMuPtulWWOizt72qxVVKiWZFddpWpv77lHZXP9/FSbsUceUdctXar+TJ+upuod76WXVNu06dPVAI0mVNdY0TQNo0HjP2uPsi+tkA9vOI/yKisPzd/BnaOjeXh8N1ycHG3LBnTy5uPV/9/efYdHWaX/H3+fdNIrJIQQIITeCR0pomBDbCiyNhA7dtfVVdeyuuvq17a67lpYBd2fotjAioggCNLB0AlNCCUJLRBSSOb5/XEGCCFUk8yE+byua65Mnv7cmST3nLnPOeso6t6TkClT7LkWLrTX3r07pKZCair5A89n7xOTAViyaTdf/rqVW/o0Oaret15kCDl7i1m8aTejxs4n0N/QtG44LRIjSIkN5YmLW9MpNYaHPvmV/SVl9Gue8LvDLCIiUhWUDEvV+PprW/faqJFN4Nq0sS2bTz4JO3Yc3u7SS21rZDkfzdvEgTKHKSu28/4vGxl1VpND63Lvuh/eX3hoyl2A5vUiMAZWbdvLecOG2WT4ww/h0Uer+y5PztNP2wdAs2Z2dAfHsd8fHNO3Z08bsw4d4M03K59aOiQE3nrrqMWO4/DJwmzWbN/LnQPSGfnuPBIjQ3hgYHNemrKac1vVY1DrRIwxnNuqHmHBR/+ad2wYQ6nLYWn/wWT87SE7QcYbb9jOc+VGl9i6+3BpxeeLswHoVclUyQkRwazN2ccd/1tIeEgAOwtKWLJ5D3++oMWhbS5uX59WSRFMXLyF/i3qnjCMIiIiNUHJsPx+M2faVtB27WDqVFsWkJ1tP/ovKLAdw5KS7ONgjSu2RXNdXgE/rNzOtd1TWZe3j1enZtE2OQo/P8P2/CIW/bYbsC2PB9UJ8ic1NpSV2/LhnHRbUvDRR96TDE+2LamMGAFjxtjSiD177OxtX39tx+EdOxb+/GfbSS00FIBnv1nJ54uyiY8IYnjXVIZ3a3jUoX/bsZ+HP/uVn7PsG4xJS7awZU8RQQF+bM8vwt8YnhrS+lAJQmWJMNiWYYBFTTuS4XLZ0R/ef9+2Yh8cOQLYsrvw0PNPF9pkuF2DKCqqFxnCjDV5BPn78fGtPXjh+9UY4MbeTY7YrmndCO4b2PxkoigiIlIjlAzL77NhAwwebGtWv/nmcH1scrIdM7iCPYUHeOGLpcxdv5N1uQWUlLnwM3Bpp2RiQoO46o3ZXPXmL0fsE+BniA0NOmJZ88QIVm3ba7+58EI7K11ODtT1cItjaSlkZsI999haZrAt4bGxcNVVNhnu2dOWdHz00aHdMjfv4Y2f1tKpYQyFJWU8+nkmzRPD6ZwaS87eIh79bCm90+P529crCPTz45lL25AcXYdRY+fj72coKXUxZ/1Onry4NUlRdU54mfHhwTSOD+MXVx1uCg+HJ56A0lJmXXwto//6PR/f2oO0hHCy3clwZEgA+UWlxIUFEV3hZwGQ6H6z8tjgVrRPieadG7rgZ1BdsIiIeD0lw3JYTg6EhdnHyZg4Ef70J9vy+c03J5WIXjdmDsu25HNWejx9myfQrG4E7VOiaFrXjqLwxehezMraQXxEMF/9uoWP5m8msk4gfn5HJlUtEiP5btl2bh43n9f79LUv5OnTD49EUd1cLnjkETss3IUXHo7ZsmVQWGjHT65o8GDb6nrRRUcs3lN4gLs/XERcWDDvjOiCAc5/ZQb3jF/MN3f34fUf1zJ5+XYmL99OalwoH97c/VDCO25kV0rKXDz0SSaJUSFc0z31pG/hrPR4JizYTEn/swmaNBGSkvhsZyA7C0r404RfuW9gM1Zv34u/n+H5oe255b0FZDSKqfRYw7s1JK1uGJd0SAbA309JsIiI1A5KhuWwevVs57Zffz3xtrNnw2WXQePGMH68/ej/BDbt3M+SzXt45IKW3NSn8u2ToupweWc7La/L5fDR/M3sLCg5arsL2iYxe90OJi/fzlet2zIkPBymTau5ZHjy5MMzvQUG2hKD9u0PL+va9eh9YmJsx8JyraVlLoc7P1jEbzv3879R3YgMsaNvvHRVB656YzZ3fbCImVl5tE2OInt3Ic9e1u6Ilt+e7vrdT2/vSVhwwCkloX3SExg3eyPze19Iz0kT4eyzWZNjR4CYv3EXw9+aA0BydB0GtU7kh/v7Ehd2dKswQP3oOlzaUdMpi4hI7aNkWKxCd21oZubJbT96tB3qa/78I4YOy8kvYk/hAT5ZmM0956QfHhkCmL3O1rn2aXZyIwkcqxUSbJnEhzd155yXpvPWrI0M6d3bJsM1weWCF1+0M7298IKdOe/aa+2oGQEBcMstkJZG7t5iig6UkRIbemjXAy6H//tuJQH+hlv7pvHKlDX8tDqXv1/Wlm5NDtfqdmkUy+39mvLaj1nEhwfz1nUZ1I0IPqqF/KD60ScujaioR1ocgf6G6clt6OnnR9F5F7BsxR5u7ZvGyF6NWLV9L6u27T00HnBagsYFFhGRM4+SYbHWrz/8fNOmQ+PWVmrlSjsM18svH5EIvzF9LX//ZiVBAX6HZkAb2asxefuKyd1bzKQlW4gLC6JZvZNLqiJCArnz7Ka0SoqsdL2fn+H6Ho14fOIyVvQeRMtH763+uuHCQrjhBjs273PP2RE0evSA226DAQNgxAj2RMaxZuMu/uQek3dQ60Tiw4OJDg2kQUwob/xkp5T+aXUemdl7uKFnI67uenRnubvPScflOJzfJonEqJCj1v9eYcEBdE6N4ae8Uh5eu5aFpeEcWDqHzqkx1I0MoW5kCGelawg0ERE5sykZFmvt2sPPZ8yA4cMr3+7AAXjtNftRf7mShLGzNvD3b1bStVEsxWUu6gT68erULF6dmnXE7kM7NzilTlX3n2DkgYvaJfHXL5fzeWI7WoKtGx4yBM4/H/r0gb/8pfJhy07H9u322HPn2lEgHnjALk9PhylTDm324hdLGTt7I2BbwRdv2s2ewgPsLSp1T5EMg1ol8u2ybfRuGs+jF7as9HSB/n48eF6LStdVlb7N6vKPb1eSE5vIfz/LJDo0kJ5pcSfeUURE5AyhZFisLHfS6u8PDz5oRzsoX/e6fz+8/jq88gps3mwT4fr1ARg/7zcen7iMc1vV4/U/dCLQ34+C4lImLtlCYUkZCRHBhx6p5UoGqkJceDB9miXw5bZ8HgoPx0ybZkd0mDrVPvLzbSlDVbjlFltP/ckndrzkY5iz3k5B/ND5Lbi1b9qh5fd9tJhPF2bTIjGCZy5tQ7N64Yzs3ZgAf7+qub7T0KdZPP/4Ft6dtYEpK3K479xmxxyOTURE5Ezkuf/C4l3WrrUlDwsW2A5hZ50F77xj17lcdliwP/7RtoJOmmQnuQBmr93Bw59m0qdZAq8N70igO7ELCw7g6q4NGdm7MYPb16d7kzjSEsKrJfHr36Iu2buL+O3sC2zd8Guv2escPdrW9n788e8/iePAzz/D1VcfNxEuLCljTc4+7jy76RGJMMAFbZIAWwsdFx7MfQObVzpMWU1qmRhJfHgwb8+wZTJXdFYnOBER8S1Khn3dsmUwbx5kZTGt23msTmxiO8X16QMjR8Jjj9lZ5b780rYKT50KF12EYwxPTlrGDe/MpVF8GK//odMR0/3WpIMf6//ceQAsXw6zZsGoUbamuVUreOopm9D/HtnZkJcHnToBsL+klMGvzuRPE35l8rJtlJbZ42dm76HM5dAhJfqoQ5zVLJ6Brep51agLfn6GPunxlJS5SEsIO62OeCIiIrWZkmFfVVxsZ2xr3x769oVp07ih47UMfOknOxbuN99Ay5Z2WuEDB+wMZXfeeWj3937ZyDs/b+CCtkmMHdGVcA9+tN4kPox6kcHMiml8eOEVV9iSj0cegaVL7ZjIixfDHXdAWdmpn2ThQvvVnQx/sjCbzOw9jJ+/iZvfW8BjXyzj39PW8sDHS/AzVJoMBwf48+Z1GXROPfYoGZ5wcHQPdZYTERFfpGTYF61fD507wzPP2PKHwEB2hx9O3spcjh0i7LHH7IJnn7Ud6twd0Q6UuXhlyhp6NY3jhaHtjxg6zBOMMfRKi2f2bgdXeAR06HB43OMrr7T1z08/Da++auuelyw59ZMsWmTvv107sncX8sb0tbRrEMWix85leLeGfDD3N/7x7UoSI0N449oM4sKDq/Ymq1G/5gm0axDFpR2TPX0pIiIiNU49ZXzRk0/aaZS/+gouuABmz2Z9bhHM2g/A8i35tG0QBcOGQZs29lHOzKw8dhSUcF2PRscc97am9UiL49NF2ax++U1aNE1izfa9JEaFEBESCA8/DDfeeHgM5WnTDrXwnrRp06B1a+blFnPrewsoKXXx4pUdiAkL4i8XtaJN/Si6N4mlSS0cizc6NIiJo3t7+jJEREQ8Qi3DvmbXLjtj3DXX2EQYoEcPNqSkH9rki8XZOI5jW0Lbtj2iRXjSki387asVRIYE0K+593ysfnAmtlnNu5HdvivnvvQTvZ6dym879tt7bdjQTooBpz45x9atMH06X10yiuFv/UJUnUA+u6MXXRvHAhAS6M/wbg1rZSIsIiLi65QM+5oxY6CoyA4TVs6GvP34GRjYqh5vz1zPveMXs7+k9ND6mWvy6PPcj9z5wSJKylw8d0V7j3WYq0xydB0axYUya20e42ZtACC/qJRrxswhr8SxJR9BQTBoEPz0k62DPlkTJoDj8K+IVqQlhPPZ7b0OzcomIiIitZuSYV9SVGSHGjv7bOjY8YhVG3YUUD+6Dv+5pjP3n9uML5Zs4ZJ//czWPYV8MPc3bnt/AWHBAbx9XQZT7+/HeW0SPXQTx9YjLZ4563by4bxNXNg2ic/v6MXWPYW8MmWNHV1i61a4+WbYs8cOk3ayPvyQog6dWL37AGe3qEtUaGD13YSIiIjUKNUM+4rCQrj4YpsQvv8+ANm7C3lq0jLKXA5rcwtoHB+Gn5/hzgHptE+J5sax87jonzPZUVBC4/gw3h3RhQYxnu0sdzw90+L4YO5vAFzeOZkOKdFc0TmF8fM2cVu/NOIjo/mtU2+aBgfDF19Av34nPujGjTBrFiufeonSAod2DaJOvI+IiIjUGmoZ9hVvvGGnDH7rLcr69eedn9cz8MXp/LgylykrclifV8Dwrg0Pbd6nWQLDujRkR0EJl3ZMZur9fb06EQbbiQ4gIiSA3k1tPfMd/dNwcHh9WhZvz1zH+W/NJ3fQYHjvPfj0UzvF8vF89hkAmR1sB7M2yUqGRUREziRqGfYFxcXw/PN2POFRo3hnxjqe/moFfZsl8PQlbfj39LWUlLqOKn24+5x0AvwNd/RvijHeMWrE8cSHB9O7aTwtEiMICrDv8xrEhDI0w7YOp8SEcqDMYfZND3DxskVw+eV2x/r17egSN9xg64nvuw9SU+26JUsgKYlFhQHEhAaSrEkpREREzihKhn3BuHGwZQu8+y5lLod3Z22ga+NY3h3RBWMMf7u0baW7xYcH8/jg1jV8sb/P+6O6HbXsjv5N+Xj+JtblFQAwq7gOF2dmwpw5dvzghQvtCBNXXGF38Pe3tdUAq1dT3KIVU5ZvZ0DLerXiTYGIiIicPJVJnOlKS+2kGV26wDnnMHVlDpt3FXJ9j0Y+k9glR9fhyowUAOpHhfDz2jyckBBbM3zvvbZkYulSGD0agoPh228P77xqFT+26kV+USlDOtT3zA2IiIhItVHL8JnA5bKtmxMn2nGER4w4PKnE+PGwbh288AIO8NqPWTSIqcPA1vU8esk17cHzWtAhJZqiA2U89sUysnL2kV4v4vAGUVF2hrr0dLj7bhvP1FTYsYMv4loSHxhEb/dYxiIiInLmUMtwbbRzJ3z5JVx3HXz+uZ1QoksXO73ymDHQqxdMn26T5L//HVq3xhk8mJe+X82STbu5o39TAv1960cfVSeQoRkpnNvK1kV/t2xb5RtefjnExECPHnDnneQHhfJDSTgXtatPgI/FTERExBfov3tNycmxH8cvWHB6+zuOHRt36FBISIDB7hERLr3Urhs71p5j40bbonn99Xa0hGXL4OGH+XpZDv+cmsVlnZIZ2rlB1d5bLZIYFUKHlGg+WZjNpp37j94gORmWL7dJ8Qcf8F2zHpQ4RiUSIiIiZyjjOI5HTpyRkeHMnz/fI+f2iKuvhg8/hHr1bGc2v1N4H+I4tnPXp5/aVstRo+Css2ynrxdfhLffhhtvPLz9pEl2TGGAJk1wrVjJef+aRZnLYfK9ffH3841a4WOZunI7t/9vIUUHXMSHB9OqfiStkiK5rFMy8zbsZHD7+kSGBML33/PI9GwmBSSx5PGBPlNjLSIicqYxxixwHCejsnWqGa4JhYU2QfX3t+PaLlwIGZX+PI5UWgrDh8PKlZCZCX/8Izz+OISF2fUDB8I559gphsu78ELo1s2e6403+HZVHqu37+OVYR18PhEGOLtFPSbf05cpK7azfGs+y7fkM2btOv4zfS0A2/YUcf/A5nDuuaxdP5umpS4lwiIiImcoJcM14dtvoaAAPvjAthCPGGFbdPv3h7w8W/rQvbv9eL5DB9i923Z6W7kSNmywrcgpKfDXv9rRDg4KDobzzz/6fH5+MHs2AC4H/vnPGTRJCOOidvqo/6CGcaGM7N340Pebdu7nsS+WMm1VLlNX5thkGFibW0C/ZgmeukwRERGpZkqGq1tZGTz9tK1FvfxyW94wY4Zt1Y2Pt2UTy5bBzJm2Fvjrr20pRJMmtlPcX/4CrVtDaOiRiXA5juOwNncfP6zIYW9RKZd0rE/TunakhMnLtrJy215euqq9WoWPIyU2lHdHdOU/09fy7DcrWbN9L3UjQ8jdW0xa3XBPX56IiIhUEyXD1e3f/7ZlEePHQ2AgTJhga4Y3bLDLvvrKthL36GGHQ3O5ICTkuIfcvb+ERZt2823mNga1qceTk5azcYftDObvZ/jvz+sZN7IrnRrG8MoPWTSJD2OwWoVPyoVtk/jnD2u4+LWfubBdEgBpCUqGRUREzlTqQFedtm2D5s2hWzdyJkzkx1U5NKsXQceGMad8KMdxmL1uB+NmbWTy8m243D82Y+xMcfeck07/5nXxM4ar3/qFA2UuHjq/BaP/3yJevLI9l3Xy3REkTtXmXft5+ssVfOsefm3q/X1pooRYRESk1jpeBzolw9Xpmmvg448hM5M75u3jq8ytxIUFMfeRc066ZMFxHBwHRrw7j+mrc4kODeSqjBQ6p8Ywf+Mu3vxpHc9d0e7QDGsAP67MYcS78wBoFBfKlPv6aozc0/DT6lwys/dwe780daATERGpxTSaRE1bvNh2isvKgsceY19qE35473siggPYUVDC4k276Jwae8LDFB0oY8Q789iwo4Cte4q4a0A6t/dLIyTQH4D+LerSr3kC3RvHHbFfv+YJ9G4aT96+Yp4a0kaJ8Gnq0yyBPuo8JyIickZTy3BVKyyEli3tsGh33QV33cVnK/K4d/wS/ntDBqPGzicsKICHLmjBwFaJFBSX8oe359AhJZqsnH20SIogv/AAm3YVsnnXfopLXQQH+FE/qg6T7+2jxFZERETkFKlMoiYtXgwdO8L//gfDh1NYUsbAl6cT5O/H9/f25aZx8/lhZQ4AfgaiQ4PYV1xKSamLVkmR5Owtpm5EMCmxdUiJCaV3ejxN4sMJDDAkRdXx8M2JiIiI1D4qk6hJ22ynK1JTAXj5h9Vs2lnIhzd3x8/P8O9rOlNcWkb27kK+ztzG9FU53NYvjc6pscSHB6k2VURERKQGKRmuatu326+JiSzbsoe3Z6xnWJcUujexdb1BAX4EBfjRIjGQFomR3HduMw9erIiIiIhvUwFqVXMnw+sDI3j400xiQoN4+PyWHr4oEREREamMWoar2rZtfNBlMA+/boc2e/XqjkSFBnr4okRERESkMkqGq1he7m4e6zeKs9Lj+fMFLWmZFOnpSxIRERGRY1CZxO81dSqMHQslJQBszD9AqZ8/I3s1ViIsIiIi4uWUDJ+MnBx45hn7taLRo+GGG+y0y2PGkJtfCEBCRHDNXqOIiIiInDIlwyeyZAl06QKPPgo9e9pZ5Q7avx9WrYIhQyAhAUaNInf3fgDqKhkWERER8XonlQwbY84zxqwyxmQZYx6qZH2wMWa8e/0cY0yjqr5Qj1i1Cnr3hrIyGDMG9uyBHj1g7ly7PjMTXC7bMjxnDtx0E7nhMRjHITYsyKOXLiIiIiIndsJk2BjjD/wLOB9oBVxtjGlVYbMbgV2O4zQFXgL+UdUXWq2++w4GDYJrroFdu+wylwuuvRaCgmD2bBg5EmbNgshI6NcPhg2D7t3tth07gjHw7LPkNmlJXLDRtMkiIiIitcDJZGxdgSzHcdY5jlMCfAgMqbDNEGCs+/kEYIDx5qnUXC44OA31uHFw4YWwejV89BG0bAnPP2+nU543D156CVJS7Lbp6TYhbt0aJkwAoNg/gOL6yXZ9bCy5Zw8iPjbCAzclIiIiIqfqZJLhZGBTue83u5dVuo3jOKXAHiCu4oGMMTcbY+YbY+bn5uae3hX/Xp9+CtHR4O8PderA9dfblt5ff4WZM6FtW3jwQbjuOmjSBIYPx3EcVmzNx+VyoF49rPotxAAACqNJREFUu92GDfD889z+4Lt0/dtUXp6ymj37D5C7t1id50RERERqiRodZ9hxnDeBNwEyMjKcmjw3YDvDDR0KGRkwcCAUF0NsLNx7LwQHQ9eu8P33tib4X/+CK6+kzM+fRz/L5IO5mxjcvj7/N7QdwcHB0KABrvvu55cnvqOOvx8vT1nD2zPWs6+4lMs6VXyvICIiIiLe6GSS4Wwgpdz3DdzLKttmszEmAIgCdlTJFValdu3gzTdtvW9Y2LG369oVunbF5XK4d/xiJi7ZQp9mCUxasoWc/CLevDaDqNBA1u8ooKCkjMcvbk3b5Cju+2gJK7bmExygemERERGR2uBksrZ5QLoxprExJggYBkyssM1E4Hr38yuAqY7j1HzL74kYAzfeeFQi7DgOs7LymLHmyNKNLzO3MnHJFu4/txnjRnbllWEdWPjbLq74zyw+mr+Jxz5fCkDb5ChaJkXy3OXtADTZhoiIiEgtccKWYcdxSo0xo4HvAH/gv47jLDPGPAXMdxxnIjAGeM8YkwXsxCbMtcbr09by/HerCPAzTBzdm1b1I3G5HF6cvIoWiRHc3r8pAEM6JJMQEcwt7y3gwQm/Htq/ad1wANo2iGLuIwOIC1PNsIiIiEhtcFI1w47jfA18XWHZX8o9LwKGVu2l1Ywyl8O42Rvo2iiWdXkF3PzefP5zTWcKD5SxYcd+Xr6qA/5+hwfG6JkWz+d39GLxb7v5ZOFmQgL9CSw3jFrdiBAP3IWIiIiInI4a7UDnjX7OymN7fjGPD25NUlQIt72/kMv+PYuWSZEE+fsxoGXdo/ZJSwgnLSFcHeVEREREajmf7+k1YcFmouoEMqBlXTo2jOGru3rTrXEsSzbt5qz0eCJCAo+5rzEGbx5OWURERESOz+dahh3HYWZWHoH+fkSHBvLdsm0MzWhAcIA/AHHhwbw7oiufLNxM59QYD1+tiIiIiFQnn0uGl2bnc+2YuUcsu7xTgyO+9/czXJmRgoiIiIic2XwuGV6xNR+AF69sj8uBAD9Dh5RoD1+ViIiIiHiCzyXDa3L2EhTgx5AOyUeMEiEiIiIivsfnOtCtydlHWkK4EmERERER8cFkePs+0t2TZIiIiIiIb/OpZLiguJTs3YU0q6dkWERERER8LBnetb+EzqkxtK4f5elLEREREREv4FMd6BrEhPLJbT09fRkiIiIi4iV8qmVYRERERKQ8JcMiIiIi4rOUDIuIiIiIz1IyLCIiIiI+S8mwiIiIiPgsJcMiIiIi4rOUDIuIiIiIz1IyLCIiIiI+S8mwiIiIiPgsJcMiIiIi4rOUDIuIiIiIz1IyLCIiIiI+S8mwiIiIiPgsJcMiIiIi4rOUDIuIiIiIz1IyLCIiIiI+S8mwiIiIiPgsJcMiIiIi4rOM4zieObExucDGGj5tPJBXw+c8UymWVUvxrHqKadVTTKueYlq1FM+qd6bENNVxnITKVngsGfYEY8x8x3EyPH0dZwLFsmopnlVPMa16imnVU0yrluJZ9XwhpiqTEBERERGfpWRYRERERHyWryXDb3r6As4gimXVUjyrnmJa9RTTqqeYVi3Fs+qd8TH1qZphEREREZHyfK1lWERERETkEK9Oho0xKcaYH40xy40xy4wxd7uXxxpjvjfGrHF/jXEvb2GMmW2MKTbGPFDhWHcbY5a6j3PPcc55njFmlTEmyxjzULnlo93LHGNMfHXdc3XxpliWW/9PY8y+qr7XmuBN8TTGzDDGLHY/thhjPq+u+65OHorpf40xOcaYpRWWV3rO2uY0YvoHY8yvxphMY8wsY0z7csc67u9zue2udx93jTHm+nLLnzHGbKqtv/MHeVNMy62fWPE1XFt4SzyNMRHl/o4uNsbkGWNeru77rw4eium3xpjdxpgvKyxvbIyZ495/vDEmqLru+3dxHMdrH0AS0Mn9PAJYDbQCngMeci9/CPiH+3ldoAvwDPBAueO0AZYCoUAAMAVoWsn5/IG1QBMgCFgCtHKv6wg0AjYA8Z6OTW2OpXt9BvAesM/TsTkT4lluu0+A6zwdn9oQU/e2fYBOwNIKyys9Z217nEZMewIx7ufnA3NO8fUXC6xzf41xPz94vO7u66mVv/PeGFP3+suA/1fxNVxbHt4Wz3LbLQD6eDo+tSGm7m0HAIOBLyss/wgY5n7+H+A2T8ensodXtww7jrPVcZyF7ud7gRVAMjAEGOvebCxwiXubHMdx5gEHKhyqJfaHu99xnFJgOvYPSEVdgSzHcdY5jlMCfOg+F47jLHIcZ0NV3l9N8qZYGmP8geeBB6vwFmuUN8XzIGNMJHA2UCtbhj0QUxzH+QnYWcmqSs9Z25xGTGc5jrPLvfwXoIH7+Qlff26DgO8dx9npPs73wHnuY//iOM7Wqr7HmuZNMTXGhAP3AU9X7V3WHG+K50HGmGbYN9szquYua5YHYorjOD8Ae8svM8YY7P+kCRXP6W28OhkuzxjTCNs6OweoV+6P6jag3gl2XwqcZYyJM8aEAhcAKZVslwxsKvf9ZveyM4oXxHI0MPFM+McIXhHPgy4BfnAcJ/+UbsAL1VBMj+dUz+n1TiOmNwLfuJ+f7N9Gn/gbepAXxPSvwAvA/lO/eu/jBfE8aBgw3nE3Z9ZmNRTTY4kDdrsbJU5n/xoT4OkLOBnud7+fAPc4jpNv32xYjuM4xpjjvmAdx1lhjPkHMBkoABYDZdV4yV7L07E0xtQHhgL9Tv3qvY+n41nB1cDbp7mv1/CymJ7UOb3dqcbUGNMf+0+xd41eaC3i6ZgaYzoAaY7j3OtOeGo1T8ezgmHAtdVw3BrlZTH1al7fMmyMCcT+MP/nOM6n7sXbjTFJ7vVJQM6JjuM4zhjHcTo7jtMH2AWsdheZHyyWvxXI5sgWpAbuZWcEL4llR6ApkGWM2QCEGmOyqugWa5SXxPPgtcRjP9L6qiruzVNqOKbHc8rn9FanGlNjTDvsm6ohjuPscC+u9PVnjOlWLqYXH2u76rgvT/KSmPYAMtx/R2cCzYwx06r2TmuGl8Tz4LHbAwGO4yyo0pusYTUc02PZAUQbYwLK7/97761aOF5QuHysB2CAccDLFZY/z5FF4M9VWP8E5TrUuJfVdX9tCKwEois5XwC2mL4xh4vFW1fYZgO1swOd18XSvV2t7EzjbfEEbgXGejoutSmm5bZtxNEd6I57ztryONWYuuOVBfQ8lddfue1igfXYjkkx7uexFbaplb/zXh7To17DteXhbfEEngWe9HRcalNMy23fj6M70H3MkR3obvd0fCq9dk9fwAl+oL0BB/gV+1HnYmz9XxzwA7AG21M81r19IrYmJR/Y7X4e6V43A1ju/mEOOM45L8D2vFwLPFJu+V3u45UCW4C3PR2f2hrLCtvUyn+M3hZPYBpwnqfjUgtj+gGwFdsJbzNwo3t5peesbY/TiOnb2Jb0g9vOP5nXX4VzjsT+Y80CRpRb/pw7xi731yc8HZ/aHtNy6xtRe5Nhr4onNvlr4em41MKYzgBygUL37/cg9/ImwFx3rD8Ggj0dn8oemoFORERERHyW19cMi4iIiIhUFyXDIiIiIuKzlAyLiIiIiM9SMiwiIiIiPkvJsIiIiIj4LCXDIiIiIuKzlAyLiIiIiM9SMiwiIiIiPuv/Az3rOBGPUAxFAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 864x576 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cxll6y7sTnUJ"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}